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Foreword 


This  report  describes  research  conducted  as  part  of  the  Navy  Personnel  Research  and 
Development  Center’s  Distributed  Training  Technology  (DTT)  project.  The  DTT  project  is  part  of 
our  Classroom  and  Afloat  Training  research  program  and  falls  under  the  Education  and  Training 
project  (L1772)  of  the  Navy’s  Manpower,  Personnel,  and  Training  Advanced  Development 
Program  Element  (0603707N).  The  work  was  performed  under  the  sponsorship  of  the  Bureau  of 
Naval  Personnel.  The  research  evaluated  training  strategies  and  technologies  to  extend 
videoteletraining  (VTT)  beyond  traditional,  lecture-based  courses. 

The  research  investigated  the  feasibility  of  using  videoteletraining  to  deliver  a  Quality 
Assurance  course  that  contained  a  laboratory  where  students  used  computers.  The  findings  have 
direct  implications  for  the  design  of  future  distance  education  systems  in  the  Navy  and  elsewhere. 

The  recommendations  in  this  report  are  intended  for  use  by  the  Chief  of  Naval  Education  and 
Training  in  developing  policy  for  the  application  of  VTT  in  the  Navy. 


P.  M.  SPISHOCK 
Captain,  U.S.  Navy 
Commanding  Officer 


MURRAY  W.  ROWE 
Technical  Director 
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Summary 


Problem  and  Background 

Many  Navy  personnel  requiring  training  are  geographically  separated  from  training  resources. 
Videoteletraining  (VTT)  enables  an  instructor  to  teach  multiple  classes  at  different  geographic 
locations.  VTT  has  been  an  efficient  and  cost  beneficial  way  to  deliver  training  and  is  in  operational 
use  by  the  Chief  of  Naval  Education  and  Training  (CNET)  within  the  CNET  Electronic 
Schoolhouse  Network  (CESN).  VTT  has  been  used  for  lecture-based  instruction  and  additional 
cost  savings  could  be  achieved  if  other  types  of  training  such  as  student  laboratories  could  be 
delivered  by  VTT.  A  Quality  Assurance  (QA)  course  presented  a  challenge  to  deliver  by  VTT 
because  it  contained  a  laboratory  where  students  learned  to  use  a  computer  program  to  produce  a 
QA  report.  This  required  that  remote-site  students  learn  to  use  the  program  without  the  physical 
presence  of  the  instructor  and  required  the  use  of  existing  VTT  classrooms  not  normally  equipped 
for  a  computer  laboratory. 

Objective 

The  objective  of  this  research  was  to  evaluate  the  feasibility  of  using  videoteletraining  to 
deliver  a  Quality  Assurance  course  containing  a  computer  laboratory. 

Approach 

The  approach  to  delivering  the  course  by  VTT  included  several  adaptations  of  the  laboratory 
procedures.  An  instructor  software  demonstration  was  developed  to  better  prepare  remote-site 
students  for  performing  the  computer  laboratory  with  less  assistance  from  the  distant  instructor. 
Portable  laptop  computer  systems  and  a  wireless  printer  network  were  used  by  remote-site  students 
so  that  the  laboratory  could  be  conducted  within  existing  VTT  classroom  facilities.  An  existing 
computer  classroom  was  used  by  local-site  students  during  the  computer  laboratory  so  that  new 
computers  were  needed  only  for  the  remote  site. 

The  evaluation  of  the  course  compared  students  in  two  primary  treatment  groups:  (1)  a  VTT 
local  classroom  with  an  instructor  and  students,  and  (2)  a  VTT  remote  classroom  where  students 
were  connected  to  the  local  classroom  by  a  two-way  audio  and  video  VTT  system.  There  were  48 
local  and  52  remote  VTT  students  during  four  experimental  convenings  of  the  course.  Training  was 
transmitted  from  San  Diego  to  the  remote  site  in  Pearl  Harbor,  Hawaii.  The  two  VTT  treatment 
groups  were  compared  in  terms  of  student  questionnaire  responses,  an  observer's  tally  of  instructor- 
student  interaction,  and  performance  on  a  quiz  covering  facts  about  operating  the  computer 
program  used  in  the  laboratory.  A  comparison  of  student  performance  on  an  objective  final 
examination  included  a  third  treatment  group  of  133  students  who  had  previously  been  taught  in 
traditional  classrooms. 

Results  and  Conclusions 

There  were  no  significant  differences  between  local  and  remote  students  on  the  end-of-course 
final  examination  on  the  QA  process.  These  scores  were  comparable  with  those  of  students  in 
previous  classes  taught  in  a  traditional  classroom.  There  was  no  significant  difference  between  the 


vn 


local  and  remote  students  on  the  quiz  covering  facts  about  operating  the  computer  program  in  the 
laboratory. 

Student  questionnaire  responses  generally  showed  few  differences  between  the  local  and 
remote  students.  Significant  differences  between  the  groups  were  found  in  three  topic  areas.  First, 
the  visibility  of  instructional  materials  and  training  aids  was  rated  slightly  lower  by  remote  than  by 
local  students.  Second,  access  to,  or  attention  from,  the  instructor  was  rated  lower  by  remote 
students.  Local  students  were  more  likely  to  cite  the  instructor  as  a  frequent  source  of  assistance, 
whereas  remote  students  cited  assistance  from  a  combination  of  the  instructor,  VTT  facilitator,  and 
other  students.  Third,  a  group  of  questions  indicated  slightly  greater  problems  for  remote  than  for 
local  students  on  aspects  of  the  computer  laboratory  (i.e.,  operating  the  computer  program  and 
printing  documents).  However,  the  average  difficulty  ratings  of  remote  students  were  in  a  positive 
direction  and  slightly  above  the  portion  of  the  scale  indicating  “few”  problems.  Remote  and  local 
students  indicated  a  similar  acceptance  of  VTT  as  a  method  of  instruction  on  other  questionnaire 
items. 

The  tally  of  interactions  over  the  VTT  network  showed  that  instructor  initiated  questions  during 
lectures  were  many  times  greater  than  when  students  were  engaged  in  performing  laboratory 
activities.  Remote  students  initiated  interactions  less  than  local  students,  but  remote  students 
participated  equally  when  instructor  questions  identified  a  specific  site  or  student  that  should 
respond. 

The  QA  course  was  successfully  adapted  for  delivery  by  VTT  and  it  is  feasible  to  deliver  the 
course  in  this  manner.  The  course  was  delivered  to  remote  students  without  significant  problems, 
the  evaluation  measures  revealed  small  differences  between  the  groups,  and  the  use  of  VTT  was 
accepted  by  students  and  instructors.  A  cost  analysis  was  conducted  for  the  anticipated  use  of  the 
computer  equipment  used  in  this  research  at  two  remote  sites  located  at  Pearl  Harbor  and  Bangor, 
WA.  Avoided  travel  costs  to  San  Diego  for  10  students  from  each  of  these  two  sites  were  compared 
with  the  costs  of  delivering  the  laboratory  course  by  VTT.  Avoided  student  travel  costs  for  four 
class  convenings  were  estimated  to  be  $15,504  in  excess  of  the  VTT  delivery  costs  (i.e.,  travel  costs 
minus  the  sum  of  VTT  classroom  contract  costs  plus  amortized  computer  costs).  Additional  class 
convenings  where  other  courses  shared  the  same  computers  would  reduce  the  impact  of  the  costs 
for  outfitting  VTT  remote  sites  with  computers. 

Recommendations 

The  following  recommendations  are  for  the  Chief  of  Naval  Education  and  Training  and  the 
CNET  Electronic  Schoolhouse  Network. 

1.  Delivery  of  the  Quality  Assurance  course  by  videoteletraining  has  been  successful  and  the 
use  of  this  method  should  be  expanded  to  other  sites  as  appropriate  to  satisfy  demand  for  the 
course. 

2.  Portable  laptop  computers  should  be  used  if  more  permanent  installations  cannot  be  made 
within  VTT  classrooms  for  this  and  similar  courses. 
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Introduction 


Problem 

Many  Navy  personnel  who  must  receive  training  are  geographically  separated  from  training 
resources.  An  increasingly  efficient  approach  to  meeting  this  requirement  is  needed  as  the  Navy 
downsizes  and  training  resources  become  constrained.  Videoteletraining  (VTT)  has  been  found  to 
be  an  efficient  and  cost  beneficial  way  to  address  this  issue  because  it  enables  a  single  instructor  to 
teach  multiple  classes  that  are  at  remote  geographic  locations.  The  Chief  of  Naval  Education  and 
Training  (CNET)  has  VTT  in  operational  use  in  the  CNET  Electronic  Schoolhouse  Network 
(CESN).  This  VTT  system  utilizes  an  interactive  two-way  video  and  audio  television  system  that 
allows  distant  remote  site  students  to  participate  in  the  instruction  originating  from  a  local  site 
where  other  students  are  co-located  with  the  instructor. 

VTT  has  generally  been  used  for  the  delivery  of  lecture-based  instruction.  Even  with  current 
VTT  technology,  there  is  some  reduction  in  the  quality  of  the  audio  and  video  as  compared  with 
live  instruction;  e.g.,  it  reduces  the  visibility  of  personnel  at  different  classroom  locations  and  also 
reduces  the  ability  of  instructors  and  students  to  interact  as  they  do  in  a  traditional  classroom.  These 
constraints  make  it  more  difficult  to  conduct  training  which  is  not  instructor  centered,  such  as  a 
variety  of  courses  with  hands-on  laboratories  and  learning  environments  that  are  highly  interactive. 
Additional  travel  costs  could  be  avoided  if  such  training  could  be  delivered  via  VTT  rather  than  in 
traditional  classrooms. 

A  training  course  on  Quality  Assurance  (QA)  presented  a  challenge  for  delivery  by  VTT 
because  it  contained  a  student  laboratory  that  used  computers.  It  is  a  relatively  new  course  offering 
and  student  throughput  is  sufficient  to  warrant  delivery  by  VTT.  The  three  day  course  consists  of 
lectures  and  two  types  of  student  laboratories.  Several  laboratory  periods  involve  paper-based 
written  exercises.  Students  create  a  quality  assurance  report  during  a  computer  laboratory  using  a 
program  designed  for  this  purpose.  The  ability  of  remote-students  to  learn  to  operate  the  program 
and  obtain  assistance  was  a  concern  because  these  students  were  not  physically  present  with  the 
instructor.  The  need  to  view  the  sequence  of  steps  in  operating  the  computer  program  also  required 
an  alternative  approach  to  allow  the  computer  screens  to  be  shown  over  the  VTT  system.  The  use 
of  existing  VTT  classrooms  designed  for  lecture  courses  also  required  a  solution  that  would 
accommodate  the  equipment  used  for  a  computer  laboratory. 

Objective 

The  objective  of  this  research  was  to  evaluate  the  feasibility  of  using  videoteletraining  to 
deliver  a  Quality  Assurance  course  containing  a  computer  laboratory. 

Background 

Previous  research  has  demonstrated  that  VTT  can  be  an  efficient  and  cost-beneficial  method  to 
deliver  training  electronically  to  remote  Navy  personnel  (Bailey,  Sheppe,  Hodak,  Kruger,  &  Smith, 
1989;  Barry  &  Runyan,  1995;  Rupinski  &  Stoloff,  1990;  Rupinski,  1991;  Simpson,  Pugh,  & 
Parchman,  1990, 1991a,  1991b,  1992, 1993;  Stoloff,  1991;  Wetzel,  Radtke,  &  Stern,  1993, 1994). 
This  research  on  the  use  of  VTT  in  Navy  training  has  shown  that  lecture-based  courses  can  be 
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delivered  by  VTT  without  detrimental  effects  on  achievement.  Prior  research  on  instructional 
television  also  indicates  that  student  achievement  is  not  affected  by  this  method  of  delivery  and  that 
any  initial  unfavorable  attitudes  lessen  as  a  result  of  experience  with  the  medium  (Wetzel,  et  al., 
1993,  1994).  The  major  cost  benefits  of  video  teletraining  systems  are  in  circumstances  where 
travel  and  per  diem  costs  are  avoided  by  usage  that  is  intense  enough  to  offset  the  costs  of  the 
technology.  Courses  that  are  particularly  beneficial  in  reducing  travel  costs  are  those  with  a  high 
student  throughput  and  which  are  short  in  duration  (a  week  or  less).  Historical  cost  data  developed 
by  the  CESN  from  1989  through  1995  indicate  that  the  system  reaches  the  break-even  point 
approximately  half  way  through  the  year  (i.e.,  VTT  system  costs  are  approximately  half  the  travel 
and  training  costs  that  are  estimated  to  have  been  avoided). 

The  use  of  VTT  for  a  wider  range  of  courses  would  extend  the  cost  and  efficiency  benefits  of 
VTT  beyond  the  lecture-based  courses  that  are  delivered  by  VTT.  A  substantial  amount  of  Navy 
technical  training  involves  laboratories  and  hands-on  activities.  For  example,  a  survey  of  Navy 
training  administrators  showed  that  about  three  fourths  of  their  courses  involved  some  form  of 
laboratory  and  an  analysis  of  actual  training  objectives  showed  that  procedural  learning  was  a  very 
frequent  objective  (Wetzel,  Van  Kekerix,  &  Wulfeck,  1987a,  1987b).  The  purpose  of  the 
Distributed  Training  Technology  (DTT)  project  was  to  develop  training  strategies  that  address  the 
difficulties  of  using  VTT  for  this  kind  of  course  content,  such  as  hands-on  laboratories.  As  part  of 
the  DTT  project,  the  present  study  extends  other  work  on  adapting  instructional  content  which 
presents  challenges  for  VTT  delivery.  Prior  work  in  this  area  with  Navy  courses  includes  the  hands- 
on  laboratories  in  a  Damage  Control  Petty  Officer  course  (Simpson,  et  al,  1992),  performing 
computations  and  plotting  in  a  Celestial  Navigation  course  (Wetzel,  1995),  hands-on  laboratories 
and  trouble-shooting  in  a  Fiber  Optic  Cable  Repair  course  (Wetzel,  Radtke,  Parchman,  &  Seymour, 
1996),  leadership  training  involving  small  group  processes  (Simpson,  Wetzel,  &  Pugh,  1995; 
Wetzel,  Simpson,  &  Seymour,  1995),  and  guidelines  for  converting  courses  to  VTT  (Simpson, 
1993;  Wetzel,  1996). 

The  approach  to  offering  the  QA  course  by  VTT  required  instructional  and  technological 
adaptations  to  deliver  the  computer  laboratory.  The  instruction  was  evaluated  for  changes  that 
would  enhance  its  visibility  to  remote  students  and  better  prepare  them  for  conducting  their 
laboratory  work  at  a  distance  from  the  instructor.  An  instructor  demonstration  of  the  computer 
program  was  developed  and  the  need  to  use  a  camera  to  show  the  computer  screen  was  avoided  by 
using  a  video  scan  converter.  A  permanent  laboratory  installation  within  the  remote  classroom  was 
avoided  by  developing  a  portable  computer  system  that  used  a  wireless  local  area  network. 

The  primary  issues  in  evaluating  the  feasibility  of  the  VTT  version  of  the  QA  course  were  as 
follows:  (1)  can  students  learn  new  computer  operation  procedures  at  a  remote  classroom,  (2)  how 
well  do  remote-site  students  perform  academically  in  the  course  compared  with  students  who  are 
in  the  local  classroom  with  the  instructor,  (3)  what  effect  did  the  VTT  medium  have  on  student 
attitudes  toward  the  training,  (4)  what  effect  did  VTT  have  on  the  level  of  interaction  and 
participation  for  remote  students  compared  with  those  at  the  local  site,  and  (5)  what  practical 
efforts  were  required  to  deliver  the  VTT  course  that  provided  lessons  learned  applicable  to  other 
courses.  Particularly,  it  would  be  desirable  to  apply  the  conversion  techniques  to  other  courses  that 
also  use  computers. 
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Method 


A  description  of  the  course,  the  treatment  groups  and  outcome  measures  for  the  evaluation 
study,  and  the  methods  used  to  convert  the  course  are  given  below.  The  outcomes  of  interest 
concerned  a  comparison  of  student  knowledge  on  program  operation,  final  examination  scores, 
student  attitudes  toward  the  training,  and  the  level  of  interaction  in  VTT  courses. 

Description  of  Course 

The  Afloat  Quality  Assurance  course  (J-65 1-0451)  provides  training  in  the  use  of  a  systematic 
program  to  ensure  the  quality  of  critical  materials,  components,  and  repair  procedures  for  mission- 
critical  and  safety  related  systems  on  Navy  ships.  The  training  provides  information  on  how  to 
ensure  that  materials  and  procedures  meet  the  appropriate  standards  and  how  to  get  appropriate 
authorization  for  any  necessary  deviations  from  the  standards.  The  QA  course  is  a  three  day  course 
given  at  the  Fleet  Training  Center  in  San  Diego,  CA  and  Norfolk,  VA.  The  course  is  given  about 
12  times  a  year,  with  as  many  as  20  students  per  class.  Estimated  yearly  throughput  at  the  San 
Diego  location  is  about  210  trainees.  Instructors  lecture  during  the  first  day  of  the  course  and  during 
half  of  the  second  day.  Students  fill  out  various  QA  forms  during  paper-based  laboratories  on  the 
second  day  of  the  course.  This  activity  involves  learning  to  use  reference  materials,  evaluate 
hypothetical  problems,  and  to  document  critical  information.  The  instructor  shows  correctly 
completed  forms  at  the  end  of  these  laboratories  and  the  trainees  self-evaluate  their  own  work.  A 
two-hour  computer  laboratory  is  given  on  the  third  day  of  the  course  prior  to  the  final  examination. 
Trainees  produce  a  Control  Work  Package  (CWP)  document  during  the  computer  laboratory  with 
a  CWP  computer  program  designed  for  this  purpose.  Trainees  use  the  CWP  program  to  enter  data 
that  will  be  printed  in  blocks  on  the  document  and  to  enter  paragraphs  for  narrative  sections  of  the 
report.  Trainees  print  out  the  report  for  the  instructor  to  review. 

Treatment  Groups 

A  single  independent  variable  (type  of  instruction)  with  two  states  was  used.  This  variable 
consisted  of  two  treatment  groups:  (1)  a  VTT  local  classroom  where  students  were  with  an 
instructor,  and  (2)  a  VTT  remote  classroom  where  students  were  connected  to  the  local  classroom 
by  a  fully  interactive  two-way  audio  and  two-way  video  VTT  system.  A  simulated  remote  VTT 
classroom  arrangement  was  used  for  training  instructors  prior  to  delivering  the  course  to  actual 
remote  students,  but  these  simulated  remote  students  were  not  included  as  part  of  the  formal 
evaluation.  Archival  course  final  examination  data  were  also  collected  for  students  who  received 
the  training  in  traditional  non- VTT  classrooms  for  comparison  with  the  VTT  students. 

Outcome  Measures 

Dependent  variables  (outcome  measures  to  assess  the  effect  of  the  independent  variable)  were 
in  three  general  categories:  student  academic  performance,  student  perceptions  of  VTT  and 
training  quality,  and  instructor-student  interaction.  Each  of  these  variables  was  defined  in  terms  of 
the  measures  described  below. 
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Student  Performance 


Three  aspects  of  student  performance  were  examined:  (1)  computer  quiz  percentage  correct, 
(2)  final  examination  percentage  correct,  and  (3)  percentage  of  students  passing  the  course.  The 
criterion  for  passing  the  QA  course  was  a  final  examination  score  of  80  percent  or  greater.  Student 
performance  on  computer  laboratory  assignments  was  not  used  by  the  instructors  to  determine  final 
student  course  grades. 

A  10  question  quiz  on  computer  procedures  was  developed  for  the  purpose  of  this  study.  The 
quiz  items  were  multiple-choice  and  true/false  questions.  The  quiz  covered  the  procedures  involved 
in  using  the  CWP  computer  program  for  quality  assurance.  At  the  end  of  the  quiz  there  was  space 
for  students  to  make  remarks  regarding  the  computer  laboratory  part  of  the  course.  The  quiz  was 
given  on  the  last  day  of  the  course  after  the  computer  laboratory  was  completed  and  immediately 
after  the  final  examination  was  administered.  A  copy  of  the  computer  quiz  is  given  in  Appendix  A. 

The  course  final  examination  consists  of  35  multiple-choice  and  fill-in  test  questions  that 
require  the  student  to  check  source  material,  evaluate  hypothetical  situations,  and  understand  the 
QA  process.  It  takes  approximately  90  minutes  to  complete.  The  students  take  the  final 
examination  on  the  afternoon  of  the  third  day. 

Student  Questionnaire 

A  47  item  questionnaire  was  developed  for  the  QA  course  (see  Appendix  B).  The  questions 
were  organized  in  the  following  general  categories:  (1)  instructors,  (2)  learning  &  classroom 
activities,  (3)  training  aids,  (4)  interaction/participation,  (5)  overall  course  rating,  (6)  background 
and  prior  experience  with  the  course  material,  (7)  video-teletraining,  and  (8)  open  ended  questions 
soliciting  comments  on  VTT,  what  they  liked  or  disliked  about  the  course,  and  suggestions  for 
improvements.  Most  of  the  questions  (28  items)  used  a  rating  format  where  agreement  or 
disagreement  with  a  statement  was  rated  on  a  five  point  scale  ordered  as  follows:  (1)  Strongly 
Disagree,  (2)  Disagree,  (3)  Neither  Agree/Disagree,  (4)  Agree,  (5)  Strongly  Agree.  There  were 
four  questions  that  asked  students  to  rate  the  number  of  problems  they  encountered  in  portions  of 
the  course.  These  were  rated  on  a  four  point  scale  ordered  as  follows:  (1)  No  problems,  (2)  Few 
problems,  (3)  Some  problems,  (4)  Many  problems.  There  were  two  questions  that  asked  students 
to  rate  their  skill  in  computer  use  on  a  5  point  scale.  There  were  also  eight  multiple-choice  items 
and  five  open  ended  questions,  one  of  which  was  a  second  part  of  a  multiple-choice  item. 

VTT  Interaction  Tally 

An  observer  recorded  the  frequency  of  instructor-student  interaction  between  VTT  sites  during 
the  three  days  of  the  course.  The  interaction  tally  was  recorded  for  interactions  that  occurred  over 
the  VTT  network  and  involved  recording  the  frequency  of  events  in  several  categories  given  below. 
Interactions  had  to  be  related  to  course  content.  Start  and  end  times  for  each  class  period  were 
recorded  on  the  tally  form  to  account  for  variations  of  these  periods  over  different  class  convenings. 
A  check  mark  was  entered  in  a  box  corresponding  to  whether  an  interaction  was  associated  with 
the  local  site  or  the  remote  site  for  each  interaction  category.  The  principal  categories  of  interaction 
were  whether  a  question  was  initiated  by  the  instructor  or  by  a  student.  The  tally  subdivided 
instructor  questions  (that  were  answered)  according  to  whether  the  instructor  left  the  question  open 
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for  any  site  to  answer,  named  a  specific  site  that  should  answer  the  question,  or  named  a  specific 
student  to  answer.  Student  questions  were  subdivided  into  those  that  were  initiated  by  the  student 
and  those  that  extended  into  longer  back-and-forth  conversations  and  exchanges.  Unanswered 
questions  and  reminders  to  local  students  to  use  their  microphone  were  also  recorded.  These 
categories  are  fully  defined  in  Appendix  C.  This  interaction  tally  was  used  in  previous  studies  by 
Wetzel  (1995)  and  Wetzel  et  al.  (1996). 

Subjects  and  Classes  Observed 

Instruction  was  originated  from  the  VTT  local  site  at  the  Fleet  Training  Center  (FTC)  in  San 
Diego,  CA.  The  VTT  remote  site  was  located  at  the  Fleet  Submarine  Training  Center 
(NAVSUBTRACENPAC)  FTC  Detachment  at  Pearl  Harbor,  Hawaii.  Simulated  VTT  remote 
classes  (without  an  instructor  present)  were  conducted  in  a  second  VTT  classroom  located  in  the 
same  building  as  the  local  classroom  at  the  FTC  in  San  Diego.  Traditional  (non- VTT)  classes  were 
conducted  at  the  FTC  in  San  Diego. 

Student  grades  were  compared  with  a  sample  of  233  students  who  received  similar  training 
materials.  These  students  were  drawn  from  four  convenings  of  the  VTT  class  (48  local  and  52 
remote  students)  and  from  six  convenings  of  the  traditional  class  (133  students).  For  the  classes 
studied,  the  average  class  size  was  12  students  for  VTT  local,  13  for  VTT  remote,  and  22.2  for  the 
traditional  classes.  Four  of  the  six  traditional  class  convenings  took  place  prior  to  this  study,  and 
the  student  scores  were  taken  from  class  records. 

A  subset  of  the  total  sample  of  233  students  completed  questionnaires  and  were  observed  to 
record  the  interaction  tally.  Questionnaires  were  completed  in  four  VTT  class  convenings  (48  VTT 
local,  52  VTT  remote  students).  Interaction  tallies  were  recorded  by  observers  for  four  convenings 
of  the  VTT  class. 

A  breakdown  of  the  proportion  of  officers  and  enlisted  enrolled  in  VTT  local,  VTT  remote,  and 
traditional  classes  can  be  found  in  Table  D-l  of  Appendix  D.  It  can  be  seen  in  Table  D-l  that  only 
about  1 1  percent  of  the  students  enrolled  in  the  Quality  Assurance  class  were  officers.  Table  D-2 
of  Appendix  D  shows  the  average  military  rank  for  enlisted  and  officer  students  by  treatment  group. 
The  typical  officer  student  enrolled  in  this  class  was  a  Lieutenant  Junior  Grade  or  Lieutenant.  The 
typical  enlisted  student  was  a  Second  or  a  First  Class  Petty  Officer. 

VTT  Course  Conversion 

Conversion  of  the  QA  course  for  VTT  delivery  involved  adapting  course  materials  and  training 
instructors  in  VTT  delivery  techniques.  Instructor  presentation  materials  were  converted  to  a 
landscape  hardcopy  form  and  revisions  were  made  to  enhance  the  quality  of  numerous  visuals. 
High  quality  still  photographs  of  a  mechanical  valve  were  created  so  that  identification  marks 
inscribed  on  the  valve  could  be  seen  in  demonstrations  presented  over  the  VTT  system.  An 
instructor  demonstration  of  the  computer  program  was  developed.  It  was  delivered  prior  to 
conducting  the  student  computer  laboratory.  The  demonstration  included  a  discussion  of  useful 
program  operation  tips. 
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A  phased  approach  was  used  to  refine  the  delivery  of  the  course  prior  to  delivering  it  to  actual 
remote  students.  Instructors  initially  used  the  VTT  system  to  practice  delivering  lectures.  The  VTT 
system  was  subsequently  used  to  deliver  several  dry-run  courses  where  students  used  the  laptop 
computers.  These  were  conducted  in  a  simulated  local-remote  environment  where  the  students  in 
a  class  were  divided  among  two  local  VTT  classrooms  during  actual  convenings  of  the  course.  This 
phased  approach  allowed  instructors  to  view  the  training  over  the  VTT  system  and  to  provide 
constructive  feedback  to  one  another.  This  arrangement  also  allowed  the  computer  systems  to  be 
used  by  actual  students  and  allowed  the  laboratory  procedures  to  be  refined  prior  to  sending  the 
equipment  to  the  remote  site. 

The  final  phase  of  developing  the  VTT  course  involved  going  on-line  to  actual  remote  students 
located  in  Pearl  Harbor  from  an  originating  classroom  in  San  Diego.  With  the  exception  of  the 
computer  laboratory,  both  local  and  remote  students  received  all  instruction  in  the  VTT  classrooms. 
VTT  local  students  left  the  VTT  classroom  during  the  computer  laboratory  and  used  an  existing 
computer  laboratory  in  another  building  at  FTC  San  Diego.  Remote  students  remained  in  the 
remote  VTT  classroom  at  Pearl  Harbor  during  the  computer  laboratory.  One  of  two  QA  instructors 
accompanied  the  local  students  to  their  laboratory  and  the  other  instructor  remained  in  the  local 
classroom  to  monitor  remote  students  over  the  V  IT  system.  A  VTT  facilitator  was  available  at  the 
remote  site  to  assist  students.  A  pair  of  students  shared  a  laptop  computer  if  there  were  not  enough 
computers  for  individual  use. 

Computer  Equipment 

Laptop  computers  were  used  at  the  remote  site  because  they  are  portable  and  can  be  moved  in 
and  out  of  VTT  classrooms  that  must  be  used  for  other  VTT  courses.  This  avoided  having  to 
dedicate  a  classroom  to  a  computer  laboratory.  An  infrared  wireless  local  area  network  (LAN)  was 
used  with  the  laptop  computers  to  reduce  a  clutter  of  wiring  in  the  classroom.  The  wireless  LAN 
was  used  to  communicate  with  a  conventional  personal  computer  that  acted  as  a  print  server.  A 
laser  printer  attached  to  this  computer  allowed  students  to  print  out  a  hardcopy  of  the  QA  report 
produced  during  the  laboratory. 

The  computer  equipment  and  software  used  in  this  research  are  listed  in  Table  E-l  of 
Appendix  E.  Ten  laptop  computers  were  provided  for  students  at  the  remote-site  laboratory.  This 
accommodated  a  maximum  of  20  students  when  each  computer  was  shared  by  a  pair  of  students. 
One  laptop  computer  was  provided  at  the  local  transmitting  site  for  the  instructor  demonstration. 
Students  used  the  computers  on  the  existing  desks  in  the  remote  classroom.  The  electrical  supply 
for  AC/DC  converters  used  to  supplement  the  battery  powered  computers  was  provided  through 
low  profile  power  cords  to  avoid  hampering  foot  traffic.  A  mobile  cabinet  with  lockable  doors  was 
used  to  house  the  computers  when  not  in  use. 

A  scan  converter  was  used  so  that  computer  screens  could  be  converted  to  video  and 
transmitted  over  the  VTT  system.  This  device  electronically  converts  computer  VGA  video  to 
NTSC  video.  Typical  computer  monitors  are  difficult  to  show  with  a  video  camera  because  the 
frequency  mismatch  of  these  two  devices  produces  a  crawling  band  across  the  screen.  The  scan 
rates  for  some  laptop  computer  screens  do  not  cause  this  problem,  but  they  can  cause  reflections 
from  lighting  within  the  room  and  be  difficult  to  position  in  front  of  a  camera. 
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The  scan  converter  allowed  the  instructor  to  demonstrate  the  operation  of  the  computer 
program  to  students  prior  to  their  use  of  the  program  during  the  laboratory  period.  Although  the 
resolution  provided  by  the  scan  converter  was  less  than  that  of  the  computer  display,  it  was 
sufficient  to  provide  a  general  orientation  and  to  show  the  sequence  of  operations  when 
accompanied  by  the  instructor's  verbal  description.  The  capability  also  allowed  the  instructor  to 
answer  student  questions  on  operating  the  program  during  the  computer  laboratory  sessions.  The 
instructor  would  locate  the  particular  point  in  the  program  where  the  student  was  encountering 
difficulties  and  show  that  screen  over  the  VTT  system.  The  difficulty  of  an  exclusively  verbal 
exchange  was  reduced  because  the  screen  provided  participants  with  a  visual  reference  of  the 
material  being  discussed. 

VTT  Classroom  Equipment 

The  VTT  classrooms  were  equipped  with  a  fully  interactive  two-way  video  and  two-way  audio 
digital  VTT  system.  Digital  video  was  transmitted  at  384  Kbps  using  a  VTEL  brand  386 
MediaMax  Codec  and  Multipoint  Control  Unit.  Students  viewed  all  instruction  on  two  large  TV 
monitors  located  in  the  front  comers  of  the  classroom  (40"  or  45"  diagonal  monitors).  Two  25" 
monitors  at  the  back  wall  of  each  room  showed  a  view  of  the  outgoing  video  and  the  incoming 
video  from  the  other  site.  Students  were  instructed  in  the  use  of  the  VTT  system  on  the  first  day  of 
class.  Each  pair  of  students  at  a  desk  shared  a  push-to-talk  microphone  that  was  used  to  talk  to  other 
sites  over  the  VTT  network.  Students  in  the  local  classroom  were  reminded  to  use  the  push-to-talk 
microphones  so  that  remote  sites  could  hear  them. 

The  outgoing  video  from  a  classroom  was  selected  with  a  hand-held  remote  control  that 
allowed  switching  among  several  camera  sources:  a  view  of  the  instructor  from  a  camera  on  the 
rear  wall  of  the  room,  a  view  of  students  from  a  camera  mounted  on  the  front  wall  of  the  room,  a 
videocassette  recorder  (VCR),  and  one  of  several  video  sources  located  at  the  instructor  podium. 
A  manual  six-input  video  switch  attached  to  the  instructor  podium  at  the  local  site  allowed  one  of 
several  devices  to  be  selected  as  the  outgoing  video  from  that  location.  The  primary  device  used  at 
the  podium  during  typical  lecture-based  courses  is  a  video  document  camera  on  an  easel  copy 
stand.  This  device  is  used  to  show  hard  copy  materials  or  small  demonstration  items.  The  VTT 
classrooms  were  not  equipped  with  chalkboards  or  whiteboards.  The  other  video  source  switched 
from  the  podium  during  this  course  was  the  scan  converter  used  to  show  the  screen  of  the 
instructor's  laptop  computer. 


Results 

The  treatment  groups  were  compared  on  student  performance  in  the  course,  student  responses 
on  the  post-course  questionnaire,  and  participation  patterns  from  the  interaction  tally.  The  number 
of  students  for  the  analysis  varied  slightly  over  these  different  outcome  measures  (cf.  Appendices 
D  and  F).  Results  for  all  data  collection  measures  were  available  for  the  two  VTT  groups.  The  most 
important  comparisons  in  these  data  are  between  these  two  VTT  groups  because  they  attended 
exactly  the  same  classes,  whereas  the  traditional  students  attended  separately  convened  classes.  A 
cost  analysis  that  incorporates  the  expense  for  computer  equipment  is  provided  at  the  end  of  the 
results  section. 
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Student  Performance 


Student  performance  was  assessed  with  three  measures:  percentage  correct  on  the  computer 
quiz,  percentage  correct  on  the  final  examination,  and  the  proportion  of  students  passing  the  course. 
The  ratio  of  officer  to  enlisted  students  was  approximately  the  same  in  the  treatment  groups  (11% 
were  officers  over  all  groups).  The  results  are  therefore  presented  without  distinguishing  between 
officer  and  enlisted  scores  (breakdowns  for  these  subgroups  are  given  in  Appendix  D).  The 
treatment  groups  in  these  comparisons  consisted  of  48  VTT  local,  52  VTT  remote,  and  133 
traditional  students.  The  three  groups  were  compared  for  the  final  examination  and  percentage  of 
students  passing  the  course.  No  computer  quiz  data  were  available  for  traditional  students. 

Computer  Quiz 

The  average  percent  correct  on  the  computer  quiz  is  shown  in  Figure  1.  The  mean  of  the  VTT 
local  students  (46.5)  is  less  than  one  percent  higher  than  the  mean  of  the  VTT  remote  students 
(45.7).  A  one-way  analysis  of  variance  showed  that  the  difference  between  these  two  means  was 
not  significant,  F  <  1 ,  p  >  .05. 

60 - - 


VTT  VTT 


Local  Remote 

Figure  1.  Average  percent  correct  on  computer  quiz. 

A  remarks  section  was  provided  at  the  end  of  the  computer  quiz.  Tables  F-3  and  F-9  in 
Appendix  F  give  the  percentage  of  treatment  group  responses  to  this  request  for  remarks.  Only  14.3 
percent  of  the  VTT  local  students  commented  in  the  remarks  section,  while  38.3  percent  of  the 
VTT  remote  students  made  comments  in  this  section.  A  Chi  Square  test  showed  that  the  frequency 
of  providing  a  response  was  significantly  different  between  the  two  groups  (x2(2)  =  4.44,  p  <  .05). 
Most  of  the  VTT  local  student  comments  were  positive  comments  on  the  course  or  the  computer 
program,  whereas  most  VTT  remote  student  responses  were  complaints  regarding  difficulties  with 
the  computer  software.  The  methods  available  for  printing  documents  changed  over  the  course  of 
the  study  and  the  level  of  student  complaints  may  reflect  these  differences.  For  example,  local 
students  printed  directly  to  a  printer  during  some  laboratories;  during  other  laboratories  the 
instructor  printed  the  documents  from  a  floppy  provided  by  the  student.  Remote  students  always 
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printed  via  the  LAN  and  the  infrared  devices  needed  to  be  repositioned  during  the  study  to  enhance 
reception  for  varying  student  station  locations  within  the  room. 

Final  Examination 

The  average  percent  correct  on  the  final  examination  is  shown  in  the  left  hand  panel  of  Figure  2. 
The  means  for  the  traditional  and  the  VTT  local  students  are  almost  exactly  the  same  (traditional, 
93.9;  local,  93.8).  The  average  percent  correct  for  VTT  remote  students  was  only  about  2  percent 
lower  than  the  scores  for  the  other  two  groups  (remote,  91.9).  A  one-way  analysis  of  variance 
showed  that  the  differences  among  these  means  were  not  significant,  F( 2, 230)  =  1.94,  p  >  .05.  The 
average  percent  correct  on  the  final  examination  for  all  students  was  93.4. 


Final  Exam  Percent  Passing 


Tradi-  VTT  VTT  Tradi-  VTT  VTT 

tional  Local  Remote  tional  Local  Remote 


Figure  2.  Average  percent  correct  on  course  final  examination 
and  percentage  of  students  passing  the  course. 

Percent  Passing  Course 

The  percentage  of  students  passing  the  course  is  shown  in  the  right  hand  panel  of  Figure  2. 
Students  passed  the  course  if  they  received  a  score  of  80  percent  or  better  on  the  final  examination. 
Computer  quiz  scores  were  not  included  in  the  criteria  for  passing  the  course.  Of  the  233  students 
included  in  this  study,  only  6  students  (2.6%)  did  not  pass  the  course.  Of  the  133  students  in  the 
traditional  classes,  five  students  (3.8%)  failed  to  pass  the  course.  One  of  the  52  remote  students 
(1.9%)  did  not  pass  the  course.  All  48  local  students  passed  the  course. 

The  data  for  students  passing  the  course  were  evaluated  with  a  test  of  the  proportion  passing 
(Fleiss,  1981,  d.139).  The  proportion  of  students  passing  in  the  three  groups  did  not  differ 
significantly  (%^(2)  =  2.10,  p  >  .05). 
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Discussion  of  Performance 


None  of  the  performance  measures  showed  a  significant  difference  between  any  of  the 
treatment  groups.  VTT  local  and  remote  students  performed  at  a  similar  level  on  the  computer  quiz. 
Performance  on  the  final  examination  was  at  a  high  level  for  all  three  groups  of  students.  The  very 
small  differences  among  the  treatment  groups  on  the  final  examination  were  not  significant.  The 
number  of  students  passing  this  course  was  high,  and  the  number  of  students  who  failed  was 
negligible  in  the  VTT  groups. 

These  results  indicate  that  the  QA  course  can  be  taught  successfully  by  VTT.  Performance  on 
the  computer  quiz  was  not  at  a  high  level  for  either  remote  or  local  student  groups.  This  result  may 
be  explained  by  a  combination  of  the  following:  (1)  the  quiz  contained  questions  that  were  too 
technical  for  inexperienced  computer  users;  (2)  students  knew  that  the  quiz  did  not  contribute 
toward  their  final  grade;  (3)  the  overall  emphasis  in  the  course  is  on  QA  content,  while  the 
computer  quiz  emphasized  procedures  for  inputting  and  retrieving  data. 

Student  Questionnaire 

Student  evaluations  of  the  VTT  training  were  measured  with  a  post-course  questionnaire 
(Appendix  B).  The  questionnaire  was  completed  by  a  total  of  100  students  (52  local  and  48  remote) 
from  the  four  VTT  class  convenings. 

Student  Prior  Experience 

Items  30-32  asked  students  about  their  prior  experience  with  quality  assurance  concepts  and 
the  use  of  computers. 

Item  30  asked  students  to  rate  their  agreement/disagreement  with  the  statement  “Before  this 
course,  I  had  basic  knowledge  of  quality  assurance  concepts  and  forms.”  A  five  point  scale  was 
used:  (1)  strongly  disagree,  (2)  disagree,  (3)  neither  agree/disagree,  (4)  agree,  (5)  strongly  agree. 
Student  responses  were  near  the  midpoint  of  the  scale.  The  average  for  VTT  local  was  2.81,  and 
the  average  for  VTT  remote  was  3.13.  An  analysis  of  variance  showed  no  significant  difference 
between  these  two  groups,  F(l,93)  =  1.47,  p  >  .05.  The  Pearson  correlation  between  responses  to 
Item  30  and  student  percent  correct  performance  on  the  final  examination  was  negligible  (r  =  0. 145, 
df=  93,  p  >  .05). 

Item  31  asked  students  to  rate  their  general  skill  or  background  on  using  computers  on  a  five 
point  scale:  (1)  None/Beginner,  (2)  Novice,  (3)  Intermediate,  (4)  Accomplished,  (5)  Advanced. 
Students  who  clearly  indicated  a  low  level  of  background  skill  with  computers  (None/Beginner) 
comprised  21  percent  of  both  groups  combined.  An  analysis  of  variance  showed  that  the  means  of 
the  two  VTT  groups  (local  2.81  and  remote  2.43)  were  not  significantly  different,  F(l,97)  =  2.81, 
p  >  .05.  There  was  a  weak  but  significant  correlation  between  ratings  on  this  question  and  the 
percent  correct  score  on  the  computer  quiz  (r  =  0.248,  df =  97,  p  <  .05). 

Item  32  had  students  rate  their  level  of  typing  or  keyboard  skill  on  a  five  point  scale:  (1)  Hunt- 
and-peck,  (2)  Beginner,  (3)  Intermediate,  (4)  Accomplished,  (5)  Advanced.  Students  who  clearly 
indicated  a  low  skill  level  in  typing  (Hunt-and-peck)  comprised  17  percent  of  both  groups 
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combined.  An  analysis  of  variance  showed  that  the  means  of  the  two  VTT  groups  (local  2.50  and 
remote  2.67)  were  not  significantly  different,  F  <  1,  p  >  .05.  Scores  on  the  computer  quiz  and 
ratings  on  this  item  were  essentially  uncorrelated  (r  =  0.064,  df=  97,  p  >  .05). 

The  responses  to  the  three  background  questions  (30,  31,  and  32)  indicated  that  students  had 
some  general  familiarity  with  the  ideas  and  skills  that  are  presented  in  this  course.  The  average 
student  in  this  course  rated  his  or  her  skills  a  little  less  than  “intermediate”  and  about  a  fifth  of  the 
students  gave  responses  at  the  lowest  computer  skill  level.  The  two  treatment  groups  had  similar 
backgrounds,  and  performance  in  the  course  was  generally  not  related  to  students'  assessment  of 
their  previous  experience. 

Student  Ratings 

Students  rated  a  series  of  statements  that  fall  into  the  following  four  categories:  instructors, 
learning  activities  and  training  aids,  interaction/participation,  and  Video-Teletraining  (Items  1-22, 
33-36  in  Appendix  B).  These  statements  were  rated  on  a  5-point  scale  with  a  midpoint  of  3  using 
the  following  scale  values  and  labels:  (1)  strongly  disagree,  (2)  disagree,  (3)  neither  agree/disagree, 
(4)  agree,  and  (5)  strongly  agree.  There  were  also  five  items  that  asked  students  to  rate  the  extent 
of  problems  encountered  during  specific  course  activities  (Items  24-28).  These  statements  were 
rated  on  a  4-point  scale:  (1)  no  problems,  (2)  few  problems,  (3)  some  problems,  (4)  many  problems. 

Figures  3, 4  and  5  show  the  mean  ratings  for  VTT  local  and  VTT  remote  students  for  items  that 
were  rated  on  a  five-point  scale.  Students  gave  positive  ratings  for  the  dimension  being  measured 
and  the  average  rating  for  most  items  is  slightly  above  the  “agree”  region  of  the  scale.  The  wording 
of  the  statement  in  Item  21  results  in  a  low  value  that  still  reflects  a  positive  rating  because  of  the 
wording  of  the  statement. 


INSTRUCTORS 

1 .  Instructors  were  prepared  and 
presented  lessons  clearly 

2.  Instructors  encouraged  class 
participation 

3.  Instructors  answered  student  questions 
adequately 

4.  Instructors  helped  students  who  needed 
assistance 

5.  Instructors  maintained  adequate  control 
of  class 


Disagree  Agree 

1  2  3  4  5 


Figure  3.  Student  ratings  on  instructors  (Items  1-5). 
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LEARNING  ACTIVITIES 
&  TRAINING  AIDS 


Disagree  Agree 

1  2  3  4  5 


6.  I  could  clearly  see  the  instructional 
materials  presented  by  the  instructor 

7.  I  could  clearly  hear  the  instruction 

8.  The  course  provided  skills  that  can  be 
applied  on  the  job 

9.  I  had  access  to  the  instructors  outside  of 
class  concerning  the  course  content 

10.  Instruction  and  demonstrations  on 
completing  the  forms  was  adequate 

1 1 .  Instruction  and  demonstration  of 
computer  procedures  was  adequate  for 
the  computer  lab 

1 2.  Paper  and  pen  exercises  were  completed 
without  problems 

1 3.  The  computer  laboratory  was 
accomplished  without  problems 

14.  Training  aids  were  used  effectively 

1 5.  Details  of  training  aids  could  be  clearly 
seen  (for  example,  transparencies/ 
graphics/forms) 

16.  Table  space  and  room  layout  were 
adequate  to  accommodate  the  use  of 
computers 


Figure  4.  Student  ratings  on  learning  activities  and  training  aids  (Items  6-16). 


INTERACTION  /  PARTICIPATION 


Disagree  i 

1  2  3  4  5 


17.  Interaction  between  instructors  and 
students  was  adequate  to  support  learning 
objectives 

18.  Student  participation  was  sufficient  to 
support  learning  objectives 

19.  Students  aided  one  another  in  performing 
course  exercises 

20.  Instructors  gave  equal  attention  to  all 
students 

21 .  I  hesitated  to  ask  questions  in  order  to 
clarify  procedures  or  concepts 

22.  The  ability  of  students  to  get  help  was 
adequate 

VIDEO-TELETRAINING 

33.  Instructors  coordinated  activities  among 
classroom  sites  effectively 

34.  Local  and  remote  site  classrooms 
participated  equally  in  the  class 

35.  Instructors  called  on  students  at  all  of  the 
classroom  sites  about  equally 

36.  The  VTT  facilitators  assisting  the  instructor 
were  helpful  to  students  in  the  remote  class 


B  Local  S3  Remote 


Figure  5.  Student  ratings  on  interaction,  participation,  and  video 
teletraining  (Items  17-22  and  33-36). 


A  one-way  analysis  of  variance  was  computed  for  each  item  to  determine  the  statistical 
significance  of  rating  differences  between  the  VTT  local  and  remote  treatment  groups.  A 
significant  difference  between  groups  was  found  with  only  1 1  of  the  rating  items.  The  degrees  of 
freedom  and  F  values  for  these  tests,  along  with  an  indication  of  significant  differences,  are  given 
in  Table  F-l  of  Appendix  F. 

Ratings  for  the  instructors  were  assessed  in  Items  1-5  (Figure  3).  The  ratings  for  these  items 
were  generally  high  (above  4.0)  and  there  were  no  significant  differences  between  groups  on  the 
first  four  items.  Only  Item  5  revealed  a  significant  difference  between  groups.  Although  both 
groups  gave  high  ratings  to  Item  5,  remote  students  rated  instructor  control  of  the  class  almost  one 
third  of  a  unit  lower  than  did  local  students. 

Items  6-16  addressed  learning  activities,  training  aids,  and  audio-visual  adequacy  (Figure  4). 
Significant  group  differences  were  found  with  four  of  these  items  (6, 9, 13,  and  15).  The  ability  to 
see  the  instructional  materials  and  details  of  the  training  aids  was  rated  about  one  half  of  a  rating 
point  lower  by  remote  students  than  local  students  (Items  6  and  15),  although  ratings  were  still  high 
for  both  groups.  Remote  students  rated  their  access  to  instructors  outside  of  class  (Item  9)  about 
one  half  a  rating  unit  lower  than  did  local  students.  That  differences  on  this  item  were  not  greater 
may  be  related  to  the  observed  efforts  on  the  part  of  the  instructors  to  remain  on  camera  for  a  time 
during  breaks  and  at  the  end  of  class.  Remote  students  gave  lower  ratings  than  local  students  in 
agreeing  that  the  computer  laboratory  was  accomplished  without  problems  (Item  13).  Remote 
student  ratings  were  about  three  quarters  of  a  rating  point  lower  and  just  above  the  middle  of  the 
scale.  Other  aspects  of  the  computer  laboratory  are  considered  below. 

Interaction,  participation,  and  related  VTT  issues  were  addressed  in  Items  17-22  and  33-36 
(Figure  5).  There  was  little  indication  of  problems  in  these  areas.  Ratings  for  all  but  one  item  were 
slightly  above  four  on  the  scale.  Only  two  items  yielded  a  significant  group  effect,  with  differences 
of  about  a  third  of  a  rating  unit  (Items  20  and  22).  Remote  students  gave  a  lower  rating  than  did 
local  students  to  the  statement  that  instructors  gave  equal  attention  to  all  students  (Item  20). 
Remote  students  rated  the  ability  of  students  to  get  help  lower  than  did  local  students  (Item  22). 
The  negative  phrasing  for  Item  21  produces  a  low  rating  value  which  indicates  that  students  in  both 
groups  disagreed  that  they  hesitated  to  ask  questions. 

Items  24-28  concerned  problems  encountered  by  students  during  specific  course  activities 
(Figure  6).  These  items  were  rated  on  the  following  four  point  scale:  (1)  no  problems,  (2)  few 
problems,  (3)  some  problems,  and  (4)  many  problems.  There  were  significant  differences  between 
the  group  averages  for  all  but  one  (Item  25)  of  these  five  items.  The  local  students  remained 
consistent  in  giving  a  rating  of  about  1.5  for  each  item,  thus  indicating  very  few  problems.  Remote 
students  tended  to  rate  these  items  from  one  third  to  three  quarters  of  a  rating  point  higher  than  local 
students,  indicating  that  these  students  perceived  at  least  a  few  problems  in  each  area. 
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PROBLEMS  DURING 
COURSE  ACTIVITIES 

24.  Lectures 

25.  Laboratory  exercises  with  paperwork 

26.  Computer  laboratory 

27.  Printing  documents 

28.  Operating  the  computer  program 


Problems 

No  Few  Some  Many 
12  3  4 


1.5 
1.8 

1.7 

1.8 

1.6 

2.3 

1.4 
2.1 
1.6 
2.1 


Local  ^  Remote 


Figure  6.  Student  ratings  on  problems  encountered  during  specific 
course  activities  (Items  24-28). 

The  smallest  differences  between  groups  concerned  problems  during  lectures  and  paperwork 
laboratory  exercises  (Items  24  and  25).  Of  these  two  activities,  only  lectures  were  rated  as  having 
significantly  more  problems  by  remote  than  local  students  (one  third  of  a  rating  point).  Slightly 
larger  group  differences  were  found  with  the  three  rating  items  on  the  computer  laboratory  (Items 
26,  27,  and  28).  Remote  students  rated  significantly  more  problems  than  local  students  for  the 
computer  laboratory  overall  (two  thirds  of  a  point),  for  problems  in  printing  documents  (three 
quarters  of  a  point),  and  for  operating  the  computer  program  (about  one  half  a  point).  Despite  the 
higher  reported  incidence  of  problems  for  remote  students,  their  ratings  were  still  within  the  region 
of  the  scale  labeled  “few”  problems. 

Multiple-choice  Items 

Items  29  and  37-43  were  given  in  a  multiple-choice  format  to  assess  student  sources  of 
assistance,  opportunities  for  interaction,  and  preferences  on  aspects  of  instructional  delivery 
methods.  Percentages  of  response  to  each  choice  were  calculated  for  local  and  remote  groups  and 
Chi  Square  tests  were  used  to  test  for  significant  group  differences  in  the  distribution  of  responses. 

Items  29,  37,  and  39  provide  information  on  the  source  of  assistance  and  opportunities  for 
interaction  for  students.  Figure  7  shows  that  the  pattern  of  responses  of  how  students  worked  at  the 
computer  was  similar  for  local  and  remote  students  (Item  29).  A  Chi  Square  test  showed  that  the 
distribution  of  responses  was  not  statistically  different  for  the  two  groups  of  students  (%2(3)  =  5.82, 
p  >  .05).  There  was  a  tendency  for  remote  students  to  be  on  a  team  where  the  members  took  turns 
operating  the  computer  program  more  often  than  there  was  for  local  students  (54.9%  versus 
29.7%). 
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Both  Team  One  Team  Alone 
Members  Member 


Operated  Computer  Program 

Figure  7.  Student  responses  on  who  operated  computer 
program  during  laboratory  (Item  29). 

Student  indications  of  who  most  frequently  provided  them  with  assistance  are  shown  in 
Figure  8  (Item  37).  It  can  be  seen  that  a  larger  proportion  of  local  students  (74.5%)  than  remote 
students  (43.1%)  reported  that  the  instructor  most  frequently  provided  assistance.  Remote  students 
were  more  likely  than  local  students  to  cite  the  assistance  of  the  facilitator  (23.5%  versus  0%)  and 
other  students  (21.5%  versus  12.8%).  A  Chi  Square  test  showed  that  the  distribution  of  responses 
was  significantly  different  for  the  two  treatment  groups  (%2(3)  =  16.30,/?  <  .05). 

In  responding  to  Item  39,  most  students  (71%  overall)  indicated  that  their  opportunities  to 
interact  with  the  instructor  were  not  affected  by  the  use  of  VTT  for  the  instruction.  Figure  9  shows 
that  the  pattern  of  responses  for  local  and  remote  students  was  similar  in  judging  how  their 
opportunities  for  interaction  with  the  instructor  were  affected  by  VTT.  A  Chi  Square  test  showed 
that  the  difference  in  the  distribution  of  responses  was  not  statistically  significant  for  the  two 
groups  (x2(2)  =  4.07,  p  >  .05). 
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Percent 


100 


Instructor  Facilitator  Students  Not  Needed 


Most  Frequently  Provided  Assistance 

Figure  8.  Student  responses  on  who  most  frequently  provided 
assistance  (Item  37). 


More  No  Effect  Fewer 


Opportunities  to  Interact  with  Instructor 

Figure  9.  Student  perceptions  of  opportunities  to 
interact  with  die  instructor  (Item  39). 
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Items  38,  42,  and  43  examined  student  preferences  for  a  method  of  instruction  and  whether 
students  perceived  problems  with  VTT  instruction.  Figure  10a  shows  that  local  and  remote 
students  had  similar  response  patterns  when  asked  for  their  preference  for  a  method  of  instruction 
(Item  38).  A  majority  of  students  in  both  groups  responded  that  they  would  accept  either  method 
of  instruction.  A  Chi  Square  test  showed  that  the  differences  in  distribution  of  responses  was  not 
statistically  significant  for  the  two  groups  (%2(2)  =  2.66,  p  >  .05).  There  was  only  a  slight  tendency 
for  remote  students  to  express  a  greater  preference  for  the  VTT  method  of  instruction  (a  significant 
effect  in  this  direction  was  found  in  previous  studies;  Wetzel  1995;  Wetzel,  et  al,  1996). 


Problems  with  Willing  to 

Preferred  Method  VTT  vs.  Take  Another 

of  Instruction  Traditional  Course  VTT  Course 


VTT  Either  Tradi-  More  No  Fewer  Yes 

tional  Problems  Effect  Problems 


Figure  10.  Student  preference  for  method  of  instruction,  perceived  problems 
with  VTT  delivery,  and  willingness  to  take  another  VTT  course 
(Items  38, 42,  and  43). 

Item  42  asked  whether  students  thought  there  were  more  or  fewer  problems  as  a  result  of 
delivering  the  course  by  VTT  instead  of  by  the  traditional  method.  As  shown  in  Figure  10b,  most 
students  in  both  groups  responded  that  the  VTT  method  had  no  effect  on  problems  encountered  in 
the  course,  and  both  groups  had  very  similar  over-all  response  patterns.  A  Chi  Square  test  showed 
that  the  distribution  of  responses  was  not  significantly  different  for  the  two  groups  (%2(2)  =  0.73, 
p  >  .05). 

Figure  10c  shows  that  approximately  90  percent  of  the  students  in  each  of  the  groups  indicated 
that  they  would  take  another  VTT  course  if  given  the  choice  (Item  43).  A  Chi  Square  test  showed 
that  the  distribution  of  responses  was  not  significantly  different  for  the  two  groups  (%2(2)  =  0.31, 
p  >  .05).  This  question  also  asked  students  to  “please  explain”  their  response  and  these  written 
comments  were  clustered  into  related  categories  (Table  F-3  and  F-4  in  Appendix  F).  About  half  of 
the  students  in  each  group  responded  to  the  question.  There  were  only  slight  differences  between 
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the  groups  in  offering  comments  that  were  positive  (30%  local;  38.5%  remote)  or  negative  (26% 
local;  19.4%  remote).  Positive  comments  included  saving  money  or  travel,  interacting  over  the 
VTT  system,  and  liking  or  enjoying  VTT.  Negative  comments  cited  a  preference  for  traditional 
instruction  and  VTT  equipment  or  procedural  problems. 


Figure  11a  shows  responses  to  Item  40  where  students  chose  between  a  convenient  VTT 
training  location  and  an  inconvenient  traditional  training  location.  The  majority  of  students  in 
both  groups  indicated  they  would  prefer  a  VTT  course  near  their  home  port  over  a  traditional 
course  that  was  farther  away  from  their  home  port  A  Chi  Square  test  showed  that  the 

differences  in  distribution  of  responses  was  not  statistically  significant  for  the  two  groups  (%2(2) 
=  1.29,  p  >  .05). 
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Preference  for  Differences  in  Time  Zones 

Convenient  Location  Between  Sites 


Local  £2  Remote 


VTT  Course  Traditional 
Near  Home  Farther 
Port  From 
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No  Mild  Severe 

Problem  Problem  Problem 


Figure  11.  Student  preference  for  convenient  location  and  problems 
with  differences  in  time  zones  (Items  40  and  41). 


Student  perceptions  of  problems  resulting  from  the  difference  in  time  zones  between  the  two 
sites  is  shown  in  Figure  lib  (Item  41).  The  two  sites  differed  by  three  hours  during  the  study: 
remote  students  in  Pearl  Harbor  started  class  early  in  the  morning  (6:30  AM)  and  local  students 
in  San  Diego  started  class  later  in  the  morning  (9:30  AM).  Only  one  student  in  each  student 
group  responded  that  this  caused  a  severe  problem.  Local  and  remote  students  differed  in  their 
tendency  to  report  that  there  was  no  problem  or  a  mild  problem.  The  majority  of  local  students 
indicated  that  there  was  no  problem,  whereas  the  majority  of  remote  student  indicated  a  mild 
problem.  Thus,  students  who  started  class  early  in  the  morning  were  more  likely  to  indicate  a 
mild  problem.  To  compute  the  Chi  Square  statistic,  the  two  data  points  indicating  severe 
problems  were  collapsed  into  the  mild  problems  category.  The  Chi  Square  computed  on  the 
resulting  two  by  two  contingency  table  showed  that  there  was  a  significant  difference  between 

local  and  remote  student  responses  (%2(1)  =  4.08,  p  <  .05). 
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Open-ended  Questions 

Questions  44  through  47  asked  students  to  explain  what  they  liked  most  and  least  about  the 
course,  problems  encountered  during  laboratories,  and  to  offer  suggestions  to  improve  the  course. 
The  written  responses  to  each  open-ended  question  were  clustered  into  categories  that  are  given  in 
Appendix  F.  Two  types  of  percentages  were  calculated:  the  percentage  of  students  responding 
(Table  F-3)  and  the  percentage  of  comments  by  category  within  each  treatment  group  (Tables  F-4 
to  F-9).  The  pattern  of  responses  on  these  open-ended  items  was  generally  similar  for  the  two 
treatment  groups. 

Student  comments  on  what  they  liked  most  about  the  course  most  frequently  cited  the  hands- 
on  computer  training  or  learning  about  quality  assurance  (Item  44).  These  two  categories 
comprised  45  percent  of  local  and  33  percent  of  remote  student  comments.  Positive  aspects  of 
receiving  training  by  VTT  were  the  next  most  frequent  category  of  comments  (26%  local  and  23% 
remote). 

Student  comments  on  what  they  liked  least  about  the  course  were  distributed  over  several 
categories  without  a  predominant  theme  (Item  45).  Complaints  about  course  materials  were  more 
common  in  the  comments  of  local  (18%)  than  remote  (3.8%)  students.  There  were  relatively  few 
responses  and  small  differences  between  the  groups  on  the  remaining  topics  that  addressed  issues 
with  the  computer  program,  assistance,  VTT,  and  facilities.  Only  students  at  the  remote  site 
complained  that  the  class  started  too  early  in  the  day  (9.4%  of  comments). 

Item  46  asked  students  to  discuss  problems  encountered  during  the  laboratories.  Comments 
were  provided  by  about  a  half  of  the  remote  students  and  by  about  a  quarter  of  the  local  students. 
This  difference  in  group  response  rates  was  approximately  at  the  level  of  significance  (%2(1)  =  3.72, 
p  ~  .05).  In  responding  to  this  item,  23.6  percent  of  the  comments  by  remote  students  and  2.1 
percent  of  the  comments  by  local  students  concerned  problems  with  using  the  CWP  computer 
program.  This  result  closely  parallels  that  observed  in  the  remarks  section  of  the  computer  quiz 
discussed  earlier.  Of  several  other  miscellaneous  responses  to  this  question,  the  only  other 
prominent  category  concerned  a  need  for  more  instruction  and  more  time  on  the  computers  or 
laboratory  paper  work  (10.4%  local;  1.8%  remote). 

Slightly  more  than  a  third  of  local  and  remote  students  provided  suggestions  for  improving  the 
course  (Item  47).  These  comments  tended  to  parallel  those  on  what  was  liked  least  about  the  course 
and  in  most  cases  there  were  not  large  differences  between  the  two  groups.  About  a  quarter  of  local 
student  comments  concerned  improving  the  content  or  delivery  of  the  course,  whereas  only  a  tenth 
of  remote  student  comments  addressed  this  issue.  About  15  percent  of  the  comments  by  remote 
students  (and  none  by  local  students)  addressed  improvements  to  the  CWP  computer  program. 

Interaction  Tally 

Instructor  and  student  interactions  that  took  place  over  the  VTT  network  were  recorded  by  an 
observer.  Interactions  for  local  and  remote  groups  were  analyzed  by  the  type  of  class  activity  and 
the  type  of  interaction.  Data  were  transformed  into  a  rate  measure  to  adjust  for  differences  in  the 
recording  periods  for  these  activities  and  the  slightly  different  number  of  students  in  local  and 
remote  classrooms  (computations  and  data  summaries  are  in  Appendix  C).  This  rate  measure  was 
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the  number  of  interactions  per  student  per  hour  and  it  was  computed  to  reflect  the  level  that  would 
be  observed  in  one  average  class  convening  (per  class  basis). 

Interactions  were  recorded  during  three  types  of  class  activities:  (1)  lectures  on  the  first  two 
days  of  class;  (2)  student  laboratories  using  paper-based  materials  (all  on  the  second  day  of  class 
except  for  a  half  an  hour  laboratory  period  that  was  also  included  from  the  first  day);  and  (3)  the 
student  laboratory  using  computers  on  the  third  day  of  class  (only  remote  students  were  present  to 
use  the  VTT  network  because  local  students  used  a  computer  laboratory  in  another  building). 

Local  and  remote  treatment  groups  were  compared  in  terms  of  interactions  that  were  initiated 
by  either  the  instructor  or  students.  Figure  12  shows  three  types  of  interactions  that  result  from 
combining  several  tally  categories  described  in  Appendix  C.  Panel  (a)  shows  instructor  questions 
that  identified  a  recipient  that  should  respond  to  the  question.  These  questions  named  a  site  or 
individual  student  (tally  categories  2  and  3).  Panel  (b)  shows  instructor  questions  that  did  not 
identify  a  recipient.  These  questions  were  left  open  for  any  site  or  student  to  answer  (tally 
category  1).  Panel  (c)  shows  student  initiated  questions  and  comments  (tally  categories  4  and  5). 
Thus,  the  two  instructor  categories  differed  in  whether  or  not  a  recipient  to  answer  the  question  was 
specified  in  the  question  (an  answer  was  received  from  a  student  in  both  cases).  Student  questions 
and  the  instructor  questions  that  did  not  specify  a  recipient  shared  a  similarity  in  that  both  relied 
on  the  initiative  of  the  student. 
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Figure  12.  VTT  interaction  tally  results  for  instructor  and  student  initiated 
questions  during  three  class  activities. 


A  similar  pattern  is  shown  for  both  types  of  instructor  questions  in  comparing  the  classroom 
activities  (panel  a  and  b).  Instructor  questions  are  much  more  frequent  during  lectures  than  during 
student  laboratories.  The  higher  level  during  lectures  reflects  the  instructor-led  nature  of  this 
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activity,  whereas  the  instructor  asks  few  questions  when  students  are  engaged  in  laboratory 
activities  as  they  work  individually  or  in  small  groups. 

Instructor  questions  are  also  more  frequent  than  student  questions  during  lectures  when  the  two 
types  of  instructor  questions  are  added  together.  Combining  local  and  remote  groups,  there  were 
1.85  interactions  per  hour  per  student  for  instructor  questions  and  0.42  for  student  questions  (for 
all  students  in  a  class,  these  correspond  to  23.26  and  5.41  interactions  per  hour). 

During  student  laboratories,  the  relative  level  of  interaction  is  slightly  higher  for  student 
initiated  questions  than  for  instructor  questions.  This  is  particularly  true  for  local  students  in  the 
paper  laboratories. 

Statistical  significance  testing  for  this  interaction  rate  data  is  made  difficult  because  there  are 
only  four  classes  to  use  as  observations.  However,  the  raw  frequencies  combined  over  the  four  class 
convenings  can  be  compared  for  certain  aspects  of  the  data.  A  Chi  Square  test  was  used  to  compare 
the  observed  interaction  frequencies  with  expected  frequencies  that  were  derived  by  adjusting  the 
observed  frequencies  using  the  proportion  of  students  in  the  local  and  remote  groups.  These  group 
comparisons  were  computed  for  each  of  the  three  types  of  questions  for  the  lecture  and  paper 
laboratories  (where  recording  time  was  equal  between  groups). 

There  are  three  instances  in  Figure  12  where  there  is  a  significantly  lower  level  of  interaction 
for  remote  than  local  students.  For  instructor  questions  that  were  directed  to  either  a  site  or  a 
specific  student  (panel  a),  there  were  no  significant  differences  between  local  and  remote  students 
during  lectures  (%2(1)  =  0.20,  p  >  .05)  or  paper  laboratories  (%2(1)  =  0.54,  p  >  .05).  For  instructor 
questions  that  were  open  to  any  site  or  student  to  answer  (panel  b),  remote  students  responded 
significantly  less  often  than  local  students  during  lectures  (%2(1)  =  1 1.15,  p  <  .01),  but  not  during 
paper  laboratories  (x2(l)  =  1.16,  p  >  .05).  For  student  initiated  questions  (panel  c),  there  were 
significantly  fewer  questions  initiated  bv  remote  than  local  students  during  both  lectures  (%2(1)  = 
10.3,/?  <  .01)  and  paper  laboratories  (jp(l)  =  30.09,/?  <  .01). 

These  results  indicate  that  the  instructor  distributed  questions  equally  among  students  at  the  ' 
local  and  remote  sites.  However,  remote  students  responded  somewhat  less  frequently  than  local 
students  in  both  circumstances  where  students  initiated  an  interaction-either  when  initiating  a 
question  or  when  responding  to  an  instructor  question  that  was  left  open  to  answer  by  any  site  or 
student.  Although  remote  students  were  less  likely  to  initiate  interactions,  their  participation  was 
not  less  when  the  instructor  identified  them  in  questions.  Thus,  instructor-initiated  behavior  was 
important  in  maintaining  remote  student  participation. 

Other  aspects  of  the  interaction  data  summarized  in  Appendix  C  were  examined  without 
revealing  interesting  effects.  Several  effects  observed  in  a  previous  study  of  a  Celestial  Navigation 
course  (Wetzel,  1995)  were  only  slight  trends  in  the  present  study.  This  may  reflect  the  relatively 
brief  duration  of  the  QA  course.  There  were  only  slight  increases  in  instructor  and  remote  site 
student  questions  over  the  first  two  days  of  the  lecture  periods.  Unanswered  instructor  questions 
were  at  a  low  level,  changed  little  over  the  two  days,  and  were  only  marginally  higher  for  remote 
than  local  students.  There  was  a  generally  low  level  of  instructor  reminders  to  local  students  to  use 
their  microphones  and  these  declined  slightly  from  the  first  to  second  day  of  class. 
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Cost  Analysis 

The  costs  involved  in  offering  the  QA  course  by  VTT  will  be  considered  because  there  are 
additional  costs  beyond  those  found  for  the  typical  lecture  course.  Cost  saving  from  the  use  of  VTT 
are  realized  when  student  travel  costs  are  avoided  because  students  do  not  travel  and  instead  receive 
training  by  VTT  near  their  duty  station.  The  overall  costs  for  maintaining  a  VTT  system  are  offset 
to  the  extent  that  the  VTT  system  is  used  intensely.  Increased  usage  results  from  fully  scheduling 
VTT  classrooms,  courses  with  many  students,  and  short  duration  courses  permitting  more 
classroom  usage.  Laboratory  courses  with  few  students  are  a  concern  from  the  perspective  of 
offsetting  VTT  system  costs  with  sufficient  usage.  The  need  to  duplicate  laboratory  equipment  at 
remote  sites  is  also  a  concern  if  the  equipment  is  expensive  or  if  the  number  of  students  is 
constrained  by  access  to  a  limited  amount  of  training  equipment. 

A  successful  use  of  VTT  would  be  indicated  if  the  avoided  costs  for  student  travel  exceeded  the 
costs  for  using  the  VTT  system.  The  analysis  used  to  judge  the  success  of  the  CESN  system 
involves  a  comparison  of  the  contract  costs  for  maintaining  the  VTT  system  with  avoided  student 
travel  costs  that  result  because  training  is  not  available  at  remote  sites  and  students  must  travel.1  A 
similar  analysis  can  be  applied  on  a  smaller  scale  to  the  QA  course.  This  comparison  of  VTT 
contract  costs  and  avoided  student  travel  costs  is  described  below  on  the  basis  of  cumulative  class 
convening  costs  during  a  year.  Because  computers  were  required  to  enable  a  VTT  version  of  the 
course  to  be  delivered  to  remote  sites,  computer  costs  are  combined  with  the  contract  costs  for  VTT 
classrooms  in  the  analysis. 

Cost  Elements  and  Method 

The  cost  elements  considered  for  the  analysis  of  the  QA  course  are  avoided  student  travel  costs, 
the  contract  costs  for  using  the  CESN  VTT  system,  and  the  initial  costs  for  computer  equipment 
required  to  establish  a  remote  site  laboratory.  Several  parameters  were  selected  to  illustrate 
different  scenarios:  the  amount  of  travel  avoidance  for  sites  of  different  distances  from  the  San 
Diego  training  site,  whether  or  not  the  cost  of  the  computers  is  added  to  the  contract  costs  for  using 
VTT,  and  the  number  of  computers,  students,  and  class  convenings.  The  computers  represent  an 
initial  fixed  cost  that  would  be  incurred  in  other  courses  where  other  types  of  equipment  might  need 
to  be  duplicated  at  remote  sites  to  enable  the  course  to  be  delivered.  Hence,  the  form  of  this  analysis 
could  be  followed  for  other  courses.  Details  of  the  cost  elements  described  below  are  given  in 
Appendix  E. 

One  cost  scenario  to  be  examined  is  the  anticipated  future  use  of  the  computer  equipment 
supplied  as  part  of  this  research.  Four  QA  class  convenings  per  year  are  projected  to  be  delivered 
by  VTT.  It  is  planned  that  the  equipment  resulting  from  this  research  will  be  split  between  Pearl 
Harbor  and  Bangor,  Washington,  with  five  student  computers  at  each  site.  Additionally,  because 


*CESN  cost  estimates  from  March  1989  to  September  1995  indicate  a  total  contract  cost  of  $5,164,456  and  total 
avoided  costs  of  $9,174,332  ($7,108,424  training  costs  and  $2,065,908  conference  costs).  Thus,  contract  costs  are 
about  56  percent  of  the  estimated  avoided  costs,  which  leads  to  the  statement  that  system  operation  starts  to  break 
even  about  midway  through  a  year.  Excluding  the  ancillary  use  of  the  system  for  conferences,  the  contract  costs  are 
about  73  percent  of  the  estimated  avoided  training  costs.  The  average  number  of  students  per  classroom  is  about  10  or 
11.  An  early  cost  analysis  of  the  CESN  is  given  in  Stoloff  (1991). 


23 


the  two  hour  laboratoiy  is  brief,  the  possibility  of  double-shifting  the  laboratory  has  been 
considered  so  that  more  students  can  use  the  existing  computers.  For  example,  some  students  could 
use  the  computers  while  others  took  the  final  examination  and  then  these  two  sets  of  students  could 
switch  to  the  other  activity.  Double-shifting  is  a  special  circumstance  that  pertains  to  this  course 
and  it  may  not  be  possible  in  courses  with  a  longer  computer  laboratory.  Another  cost  scenario  to 
examine  is  the  effect  of  increased  numbers  of  class  convenings  on  the  relative  amount  of  travel 
avoidance.  More  convenings  could  represent  greater  usage  of  the  computers  that  are  shared  with 
other  courses.  Another  scenario  is  to  vary  the  number  of  computers  used  in  the  training,  which 
affects  the  ratio  of  students  per  computer. 

VTT  Costs.  The  average  contract  cost  per  room  over  all  CESN  VTT  classrooms  is 
approximately  $70,000.  Assuming  251  available  training  days  per  year,  the  per  day  cost  of  an 
average  VTT  classroom  would  be  $279,  and  three  days  of  use  for  this  course  would  cost  $837.  The 
three  day  class  convening  costs  would  be  $1,674  for  two  participating  sites  (one  local  and  one 
remote  classroom)  and  would  be  $2,511  for  three  participating  sites  (one  local  and  two  remote 
classrooms).  These  per  convening  costs  are  multiplied  by  the  number  of  class  convenings  in  the 
comparisons  made  below.  These  costs  do  not  vary  with  the  number  of  students  present  in  a  VTT 
classroom  during  a  class  convening.  However,  there  is  a  relative  benefit  in  travel  avoidance  to  the 
extent  that  more  remote  site  classrooms  participate  in  the  training.  This  is  because  the  cost  of 
additional  VTT  classrooms  is  accompanied  by  additional  remote  student  travel  avoidance  and  this 
reduces  the  relative  impact  of  the  cost  of  the  local  VTT  classroom  where  no  student  travel 
avoidance  is  generated. 

Computer  Costs.  Unlike  lecture  courses,  establishing  a  remote  site  for  the  QA  course  required 
initial  costs  for  the  computer  equipment  that  enabled  the  course  to  be  delivered  by  VTT.  The 
equipment  used  during  the  present  study  and  the  computer  system  costs  used  in  this  analysis  are 
given  in  Tables  E-l  through  E-3  in  Appendix  E.  The  costs  for  computers  were  amortized  over  an 
expected  lifetime  that  has  been  assumed  to  be  five  years.2  Maintenance  costs  have  been  assumed 
to  be  10  percent  of  the  original  cost  of  the  computer  equipment  for  each  of  the  five  years.  The  yearly 
cost  of  the  computer  system  is  the  sum  of  the  computer  and  maintenance  costs  for  the  five  year 
period  divided  by  the  five  years.  This  yearly  cost  of  the  computer  system  is  then  divided  by  the 
number  of  course  convenings  per  year  to  adjust  for  the  amount  of  usage  of  the  computer  system. 
This  per  convening  cost  of  the  computer  system  is  reduced  to  the  extent  that  there  are  more  class 
convenings  during  a  year  (e.g.,  doubling  the  class  convenings  halves  the  per  convening  computer 
costs).  This  adjusted  per  convening  cost  of  the  computer  system  is  then  added  to  the  per  convening 
VTT  classroom  costs.  The  combined  VTT  and  computer  system  cost  is  multiplied  by  the  number 
of  class  convenings  for  comparison  to  the  avoided  student  travel  costs  for  a  similar  number  of 
convenings  (Table  E-3  of  Appendix  E).  This  comparison  is  made  below  by  subtracting  the 
combined  VTT  and  computer  system  costs  from  the  avoided  student  travel  costs  (Tables  E-5  and 
E-6). 


2The  costs  for  laptop  computers  are  higher  than  for  more  conventional  personal  computers  (PCs)  and  mainte¬ 
nance  costs  would  likely  be  higher  also.  However,  conventional  PCs  would  require  much  more  effort  to  move  in  and 
out  of  VTT  classrooms.  Alternatively,  conventional  PCs  would  require  a  dedicated  installation  in  remote  site  class¬ 
rooms  that  would  likely  reduce  the  number  of  student  seats  and  would  require  an  added  furniture  cost  (e.g.,  glass-top 
desks  that  house  the  monitor  below  the  surface). 
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The  method  described  above  was  performed  for  four  circumstances  that  represent  the 
combinations  of  a  small  or  large  laboratory  (5  or  10  computers)  and  one  or  two  remote  sites  (a 
second  remote  site  requires  an  additional  printer  and  server  computer).  Instructor  demonstration 
computer  equipment  at  the  local  transmitting  site  is  included  in  these  costs,  but  no  local-site 
computers  are  assumed  because  local  students  currently  use  an  existing  computer  laboratory  in  San 
Diego.  In  the  projected  actual  circumstance  where  the  computers  from  this  research  have  been  split 
between  two  remote  sites  (Pearl  Harbor  and  Bangor),  the  equipment  cost  is  $42,034,  total 
maintenance  for  5  years  at  10  percent  a  year  is  $21,017,  and  the  sum  of  these  two  costs  is  $63,051. 
The  $63,051  total  is  amortized  to  $12,610  per  year  when  distributed  over  the  assumed  five  year 
lifetime.  Assuming  the  four  projected  convenings  of  this  course  per  year,  the  per  convening  cost  is 
then  $3,153.  Other  numbers  of  convenings  given  below  are  computed  similarly  and  more 
convenings  per  year  results  in  a  lower  per  convening  cost  because  of  the  greater  usage. 

Avoided  Dravel  Costs.  Student  travel  costs  were  considered  for  several  alternative  West  coast 
CESN  sites  for  classes  containing  10, 15,  and  20  students.  The  avoided  travel  costs  are  for  students 
who  would  have  traveled  or  students  who  would  have  traveled  had  the  travel  funds  been  available 
(i.e.,  a  VTT  course  offering  becomes  a  training  opportunity  when  made  available  at  a  remote  site). 
The  avoided  student  travel  costs  to  San  Diego  from  the  three  West  Coast  sites  used  in  the  analysis 
differ  substantially.  The  total  per  student  travel  costs  to  San  Diego  were  estimated  to  be  $608  for 
Pearl  Harbor,  $346  for  Bangor,  Washington,  and  $204  for  San  Francisco  (Treasure  Island).3 

The  analysis  assumed  that  two  remote  sites  participated  in  the  training  during  the  same  class 
convening  (avoided  travel  costs  for  all  individual  sites  and  their  combinations  are  given  in  Table  E- 
4  of  Appendix  E).  Two  pairs  of  remote  sites  were  selected  for  the  analysis  to  illustrate  different 
levels  of  travel  avoidance.  A  high  level  of  avoidance  is  represented  with  the  actual  projected  remote 
sites  for  this  course  at  Pearl  Harbor  and  Bangor.  For  comparison,  a  less  distant  site  was  substituted 
to  illustrate  a  lower  level  of  avoidance  by  considering  Bangor  and  San  Francisco  (Treasure  Island) 
as  remote  sites.  An  equal  number  of  students  was  assumed  for  each  remote  site.  The  avoided  travel 
costs  for  one  student  per  remote  site  for  these  comparisons  was  a  sum  of  $954  for  Pearl  Harbor  and 
Bangor,  and  was  $550  for  Bangor  and  San  Francisco.  In  the  analysis  below,  these  combined  costs 
were  multiplied  by  the  number  of  students  per  site  and  then  by  the  number  of  class  convenings  (i.e., 
when  computer  and/or  VTT  costs  are  subtracted  from  avoided  travel  costs). 

Cost  Comparison 

Table  1  shows  avoided  student  travel  costs  in  excess  of  the  costs  of  delivering  the  course  by 
VTT  (i.e.,  either  travel  costs  minus  only  VTT  costs,  or  travel  costs  minus  combined  VTT  and 
computer  costs).  The  entries  within  a  column  are  on  a  yearly  basis  as  a  function  of  the  number 
of  students  and  class  convening  during  a  year.  Without  the  computer  costs,  the  VTT  costs  are 
the  same  as  would  be  incurred  for  a  conventional  lecture-based  course,  or  if  computer  costs 
were  not  considered  because  they  were  already  available  at  a  remote  site  and  did  not  have  to  be 
procured. 


3These  per  student  costs  are  the  sum  of  airfare,  local  travel,  and  per  diem  costs.  Government  rate  round-trip  air¬ 
fare  to  San  Diego  from  Pearl  Harbor,  Bangor,  and  San  Francisco  was  $480,  $218,  and  $76  respectively.  Local  travel 
was  estimated  to  be  $20  and  per  diem  was  estimated  to  be  $108  for  four  days  (based  on  a  $27  composite  rate  adjusted 
appropriate  to  1  in  10  students  being  an  officer  at  a  slightly  higher  rate  of  $45  compared  with  $25  for  the  enlisted 
rate). 
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Table  1 


Per  Year  Avoided  Student  Travel  Costs  in  Excess  of  VTT  Costs  or 
in  Excess  of  Combined  YTT  and  Amortized  Computer  Costs 


Travel  Minus  Travel  Minus  Combined 

VTT  Costs  VTT  and  Computer  Costs 

5  Computers  Each  10  Computers  Each 
Remote  Site  Remote  Site 

No  Computers  ( 10  Total  Computers)  (20  Total  Computers) 


(a) 

Sites 

(b) 

Sites 

(c) 

Sites 

(d) 

Sites 

(e) 

Sites 

(0 

Sites 

Remote 

Number 

PH 

SF 

PH 

SF 

PH 

SF 

Students 

of  Class 

BA 

BA 

BA 

BA 

BA 

BA 

Each  Site 

Convenings 

SD 

SD 

SD 

SD 

SD 

SD 

10 

4 

28,116 

11,956 

10 

8 

56,232 

23,912 

10 

12 

84,348 

35,868 

15 

4 

47,196 

22,956 

15 

8 

94,392 

45,912 

15 

12 

141,588 

68,868 

2  Students  1  Student 

Per  Computer  Per  Computer 

15,504  -656  6,468  -9,692 

43,624  11,304  34,584  2,264 

71,736  23,256  62,700  14,220 

Double-shift  Half  Students 

Laboratory  2  Per  Computer 

34,584  10,344  25,548  1,308 

81,784  33,304  72,744  24,264 

128,976  56,256  119,940  47,220 


Double-shift  All  Students 

Laboratory  2  Per  Computer 


20 

4 

66,276 

33,956 

53,664 

21,344 

44,628 

12,308 

20 

8 

132,552 

67,912 

119,944 

55,304 

110,904 

46,264 

20 

12 

198,828 

101,868 

186,216 

89,256 

177,180 

80,220 

Note.  Sites  are  the  San  Diego  (SD)  originating  site  and  remote  receiving  sites  at  Pea-1  Harbor  (PH),  Treasure  Island  in 
San  Francisco  (SF),  or  Bangor,  Washington  (BA).  Computer  costs  are  amortized  over  5  years  and  by  convenings  within 
years  (including  10%  maintenance  per  year).  No  student  computers  are  assumed  for  San  Diego  because  an  existing  com¬ 
puter  laboratory  is  used.  VTT  classroom  costs  are  for  three  classrooms,  but  avoided  travel  costs  are  for  only  two  sites 
because  local  site  students  do  not  generate  travel  avoidance.  The  double-shift  laboratory  condition  may  apply  only  to 
this  course  because  of  the  relatively  brief  laboratory.  Additional  convenings  could  represent  more  than  one  course  shar¬ 
ing  computers. 
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TVavel  Versus  VTT  Costs.  Columns  (a)  and  (b)  of  the  table  show  avoided  student  travel  costs 
in  excess  of  the  contract  costs  of  the  VTT  classrooms  with  remote  sites  that  differ  in  the  level  of 
avoided  travel.  Training  is  transmitted  from  San  Diego  (SD)  to  the  Bangor  (BA)  site  in  both 
situations,  but  they  differ  in  the  magnitude  of  travel  costs  for  the  second  remote  site.  There  are 
higher  travel  costs  for  the  more  distant  Pearl  Harbor  (PH)  site  (column  (a))  than  for  the  less  distant 
San  Francisco  (SF)  site  (column  (b)).  The  relative  benefit  shown  in  columns  (a)  and  (b)  indicates 
that  there  is  more  travel  avoidance  with  the  more  distant  pair  of  sites  (column  (a))  and  with  more 
students  per  class.  Cumulative  avoided  costs  increase  with  the  number  of  class  convenings  during 
a  year.  These  relationships  are  also  shown  in  the  other  pairs  of  columns  discussed  below. 

Travel  Versus  VTT  and  Computer  Costs.  The  costs  of  five  computers  per  site  are  included 
with  the  VTT  classroom  costs  prior  to  subtracting  them  from  the  travel  costs  for  the  values  shown 
in  columns  (c)  and  (d)  of  Table  1.  Combining  these  computer  costs  with  the  VTT  costs  reduces  the 
relative  amount  of  student  travel  avoidance  compared  to  when  only  the  VTT  costs  are  used 
(columns  (a)  and  (b)).  With  four  convenings,  10  students  generate  sufficient  travel  avoidance  to 
exceed  the  VTT  and  computer  costs  for  the  more  distant  sites  (column  (c)).  However,  avoided 
travel  costs  for  the  less  distant  sites  fall  a  little  short  (-$656)  of  exceeding  the  VTT  and  computer 
costs  with  10  students  (column  (d)).  To  exceed  the  cost  of  computers  and  VTT  for  less  distant  sites, 
more  students  are  required  or  more  convenings  are  required  when  there  are  10  students.  When  there 
are  more  convenings,  there  are  more  students  and  there  is  also  some  reduction  in  the  impact  of  the 
cost  of  computers  because  the  cost  has  been  distributed  over  more  usages. 

With  five  computers  per  site,  increasing  the  number  of  students  increases  the  travel  avoidance 
slightly  while  the  costs  for  VTT  and  computers  remain  constant.  However,  there  is  an  impact  in  the 
way  the  laboratory  must  be  conducted  when  there  are  more  students  in  a  class.  Increasing  the 
number  of  students  in  a  class  requires  that  the  laboratory  be  conducted  in  two  shifts  if  a  maximum 
of  two  students  per  computer  is  accepted  as  a  reasonable  upper  limit. 

Although  four  convenings  are  planned  for  this  course.  Table  1  additionally  shows  eight  and 
twelve  convenings.  These  additional  convenings  could  represent  an  increased  frequency  of  offering 
the  QA  course.  Alternatively,  the  additional  convenings  could  represent  additional  courses  other 
than  the  QA  course  that  also  used  the  same  computers  (e.g.,  the  QA  course  and  one  other  course  at 
eight  convenings,  and  two  other  additional  courses  at  12  convenings).  The  greater  usage  provided 
by  these  additional  convenings  distributes  the  computer  costs  and  lessens  their  per  convening 
impact.  This  reduction  allows  the  relative  level  of  travel  avoidance  to  be  greater  than  the  combined 
VTT  and  computer  costs  (e.g.,  as  with  the  fewest  students  at  the  least  distant  sites  in  Table  1). 

The  projected  distribution  of  the  computer  equipment  used  in  this  research  among  two  remote 
sites  can  be  compared  with  having  left  all  the  equipment  at  the  original  single  site.  The  plan  to  use 
the  current  computer  equipment  to  conduct  four  convenings  of  this  course  with  Bangor  and  Pearl 
Harbor  as  remote  sites  provides  a  cost-beneficial  situation.  With  10  students,  these  sites  generate 
avoided  travel  costs  of  $15,504  in  excess  of  the  combined  costs  for  the  VTT  classrooms  and 
computer  equipment  (column  (c)).  Had  the  original  set  of  10  student  computers  remained  at  the 
one  Pearl  Harbor  site,  the  comparable  cost  for  10  students  and  four  convenings  would  have  been 
$6,212  (15  students  would  be  $18,372  as  per  Table  E-6  in  Appendix  E).  Thus,  for  an  equivalent 
number  of  students  per  site,  there  was  an  advantage  to  splitting  the  computers  so  that  they  could  be 
used  by  two  sites  and  more  total  students.  As  noted  earlier,  including  additional  remote  sites 
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generates  additional  travel  avoidance  and  reduces  the  relative  impact  of  the  VTT  local-site 
classroom  costs  where  no  travel  avoidance  is  generated. 

Increasing  Laboratory  Size.  Columns  (e)  and  (f)  of  the  table  show  the  situation  where  the  size 
of  the  remote  site  laboratory  is  increased  with  additional  computers  (10  computers  per  site,  or  20 
total).  This  size  of  laboratory  accommodates  one  student  per  computer  with  10  students,  half  the 
students  sharing  a  computer  with  15  students,  and  all  students  sharing  a  computer  with  20  students. 
The  travel  avoidance  is  positive  for  the  more  distant  sites  in  column  (c).  For  the  less  distant  sites  in 
column  (d),  avoided  student  travel  costs  fail  to  exceed  the  VTT  and  computer  costs  (-$9,692)  when 
there  are  10  students.  However,  increasing  the  number  of  student  to  15  at  four  convenings  or 
increasing  the  number  of  convenings  to  eight  with  10  students  produces  avoided  travel  costs 
sufficient  to  just  exceed  the  VTT  and  computer  costs.  Increasing  the  number  of  students  to  15  or 
20  results  in  more  students  sharing  the  10  computers  at  each  site,  but  does  not  require  double 
shifting  the  laboratory.  Compared  with  a  laboratory  of  five  computers  per  site,  there  is  less 
advantage  for  equal  numbers  of  students  and  convenings.  Increasing  the  number  of  computers  to 
10  requires  greater  usage  in  the  number  of  students  or  convenings  to  support  the  costs  for  the 
laboratory. 


Discussion 

The  QA  course  was  successfully  adapted  for  delivery  by  video-teletraining.  The  converted 
course  is  scheduled  to  be  offered  to  remote  sites  from  San  Diego.  The  evaluation  results  and 
observations  from  conducting  the  course  are  summarized  below. 

Summary  of  Empirical  Findings 

Student  performance  as  measured  by  the  final  examination  was  high  for  both  the  local  and 
remote  VTT  students  and  was  similar  to  that  for  students  in  traditional  non- VTT  classes.  The 
percentage  of  students  passing  the  course  was  also  similar  in  the  groups.  VTT  local  and  remote 
students  demonstrated  similar  levels  of  knowledge  on  the  fact  quiz  covering  the  operation  of  the 
CWP  program.  VTT  remote  students  were  able  to  complete  their  assignments  in  the  computer 
laboratory.  These  performance  results  are  similar  to  those  of  previous  studies  that  have  found  the 
difference  between  remote  and  local  site  students  to  range  from  no  difference  to  five  percent 
(Simpson,  et  al.,  1990,  1991a,  1991b,  1992,  1993;  Wetzel,  1995;  Wetzel,  et  al„  1996;  and  studies 
reviewed  in  Wetzel,  et  al.,  1994,  pp.  19-21). 

Student  questionnaire  responses  showed  few  differences  between  the  local  and  remote 
students.  Three  topic  areas  did  show  significant  differences  between  the  groups.  First,  the  visibility 
of  the  instructional  materials  and  details  of  the  training  aids  were  rated  about  a  half  a  rating  point 
lower  by  remote  students  than  local  student  on  a  five  point  scale.  Second,  access  to  or  attention 
from  the  instructor  was  rated  from  about  a  third  to  a  half  of  a  rating  point  lower  by  remote  students 
on  a  five  point  scale.  Local  students  were  most  likely  to  cite  the  instructor  as  a  frequent  source  of 
assistance,  whereas  remote  students  cited  assistance  from  a  combination  of  the  instructor,  VTT 
facilitator,  and  other  students.  Third,  a  group  of  items  indicated  somewhat  greater  problems  for 
remote  than  for  local  students  on  aspects  of  the  computer  laboratory.  These  items  pertained  to  the 
difficulty  of  conducting  the  computer  laboratory,  printing  documents,  and  operating  the  computer 
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program.  These  items  were  from  one  half  to  three  quarters  of  a  rating  unit  lower  for  remote  than 
local  students  on  a  four  point  scale.  However,  the  average  difficulty  ratings  were  in  a  positive 
direction  slightly  above  the  portion  of  the  scale  indicating  “few”  problems. 

Other  questionnaire  responses  indicated  that  remote  and  local  students  were  generally 
accepting  of  VTT  as  a  method  to  deliver  this  course.  A  majority  of  students  would  accept  VTT  or 
traditional  forms  of  instruction  and  most  students  would  take  another  VTT  course.  About  two 
thirds  of  the  students  indicated  that  VTT  did  not  have  an  overall  effect  on  problems  encountered  in 
the  course.  Remote  students  in  Pearl  Harbor  indicated  a  mild  problem  with  starting  the  class  early 
in  the  morning  that  is  probably  typical  of  other  courses  at  that  site.  The  convenience  of  a  nearby 
VTT  course  was  preferred  over  traveling  to  a  distant  traditional  course. 

The  tally  of  interactions  over  the  VTT  network  showed  that  instructor  initiated  questions  during 
lectures  were  many  times  greater  than  when  students  were  engaged  in  performing  laboratory 
activities.  Remote  students  were  somewhat  less  likely  to  initiate  participation  than  were  local 
students,  either  when  asking  a  question  or  when  responding  to  an  instructor  question  that  was  left 
open  for  any  site  or  student  to  answer.  However,  instructor  questions  that  named  a  site  or  student 
in  the  question  served  a  beneficial  role  in  maintaining  remote  student  participation.  These 
instructor  questions  were  distributed  equally  among  local  and  remote  students  and  both  groups 
responded  at  similar  levels. 

The  objective  of  this  research  was  to  enhance  VTT  system  utilization  by  extending  the  use  of 
VTT  to  a  laboratory  course.  The  results  indicate  that  it  was  instructionally  feasible  to  deliver  the 
computer  laboratory  in  this  course  by  VTT.  An  analysis  of  the  additional  costs  of  outfitting  remote 
classrooms  with  computers  also  indicated  a  net  benefit  for  the  anticipated  use  of  the  equipment 
resulting  from  this  research.  In  this  analysis,  avoided  student  travel  costs  were  compared  with  the 
combined  costs  for  the  computer  equipment  and  VTT  classrooms.  Amortized  computer  system 
costs  were  combined  with  VTT  classroom  contract  costs  because  the  computers  were  required  to 
enable  delivery  of  the  course  at  remote  sites.  The  projected  delivery  sites  at  Pearl  Harbor  and 
Bangor  have  five  computers  per  site  that  will  allow  two  students  per  computer.  These  sites  would 
generate  a  beneficial  level  of  avoided  student  travel  costs  with  the  projected  four  class  convenings 
a  year  and  10  students  per  site.  This  level  of  throughput  for  these  two  sites  would  produce  an 
estimated  $15,504  of  avoided  student  travel  costs  in  excess  of  the  combined  costs  for  the  VTT 
classrooms  and  amortized  computer  costs.  Additional  class  convenings  for  this  course  or  other 
courses  that  shared  using  the  same  computers  would  lessen  the  impact  of  the  costs  for  outfitting 
sites  with  computers. 

Observations  from  Conducting  the  Course 

Several  observations  on  the  delivery  of  this  course  are  applicable  to  conducting  other  hands-on 
laboratory  courses  by  VTT.  These  include  a  phased  implementation  to  develop  techniques  adapted 
to  training  distant  students  and  the  use  of  portable  equipment  to  accommodate  existing  facilities. 

In  traditional  laboratories,  students  are  physically  present  with  an  instructor  who  circulates 
among  students,  looks  over  their  shoulders  to  observe  progress,  and  provides  assistance  or 
instruction  as  needed.  This  learning  environment  may  need  to  be  conveyed  in  different  ways  to 
allow  VTT  remote  students  to  accomplish  their  tasks  without  the  physical  presence  of  the 
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instructor.  When  remote  students  perform  laboratory  activities  at  a  distance  from  the  instructor  they 
must  use  the  audio-video  capabilities  of  the  VTT  system  to  communicate  with  the  instructor,  rely 
on  resources  at  their  site,  or  rely  on  prior  instruction  to  work  more  independently. 

The  conversion  of  the  QA  course  was  benefited  by  using  a  phased  approach  to  tryout  and  refine 
the  delivery  of  the  course  prior  to  delivering  it  to  an  actual  remote  site  (cf.  conversion  methods  in 
Simpson,  1993;  Wetzel,  1996).  Instructors  first  practiced  using  the  VTT  system  and  then  used  the 
computers  in  a  simulated  local-remote  environment.  Several  dry-run  courses  were  conducted  with 
students  in  a  simulated-remote  VTT  classroom  adjacent  to  that  of  the  instructor.  This  permitted 
instructors  to  practice  using  the  VTT  system  and  allowed  the  lectures  and  demonstrations  to  be 
refined  as  a  result  of  the  experience.  For  example,  the  ability  to  see  inscriptions  on  a  mechanical 
valve  was  discovered  to  be  a  problem  in  showing  it  over  the  VTT  system.  High  quality  photographs 
were  developed  that  were  subsequently  preferred  by  students  over  using  the  actual  valve.  Unable 
to  circulate  among  remote  students,  instructors  also  developed  a  strategy  to  increase  student 
participation  by  requesting  them  to  display  their  paperwork  on  the  document  camera  where  it  could 
be  inspected  and  discussed. 

An  instructor  demonstration  was  developed  to  compensate  for  the  absence  of  the  instructor  for 
remote  students  in  the  computer  laboratory.  The  instructor  demonstration  was  delivered  prior  to 
conducting  the  laboratory  to  better  prepare  remote  students  for  performing  the  task  more 
independently.  The  demonstration  showed  the  sequence  of  operating  the  program  and  included 
helpful  tips  on  critical  aspects  of  the  program  that  were  typically  problems  during  the  laboratory. 
This  enhanced  preparation  of  the  students  was  used  to  offset  the  absence  of  the  instructor  within 
the  room  who  would  normally  have  been  available  to  provide  assistance  in  face-to-face  interactive 
exchanges. 

The  instructor  demonstration  of  the  computer  program  was  accomplished  by  using  a  scan 
converter  to  directly  send  a  video  image  of  the  screen  over  the  VTT  system.  This  avoided 
difficulties  with  positioning  a  camera  to  show  the  screen  and  reflections  from  lights  within  the 
room.  The  resolution  provided  by  the  scan  converter  was  less  than  that  of  the  computer  display. 
However,  the  resolution  was  sufficient  for  providing  a  general  orientation  to  the  sequence  of 
operating  the  program  when  accompanied  by  the  instructor's  narrative.  Material  with  low  legibility 
was  described  verbally  and  the  instructor  described  steps  to  be  shown  prior  to  executing  commands 
so  that  student  attention  could  be  focused  on  the  change  when  it  occurred.  A  similar  orientation 
technique  was  required  for  high  resolution  static  materials  in  a  Celestial  Navigation  course 
(Wetzel,  1995).  Nautical  publications  contained  small  print  tables  and  critical  text  was  enlarged 
and  arrows  pointed  to  these  entries  so  that  students  could  locate  entries  in  their  own  publications. 

The  capability  to  show  computer  screens  over  the  VTT  system  was  also  used  during  the  student 
laboratory  when  the  instructor  answered  student  questions  on  operating  the  program.  The 
instructor  would  locate  the  particular  point  in  the  program  where  the  student  was  encountering 
difficulties  and  show  that  screen  over  the  VTT  system.  Thus,  the  difficulty  of  an  exclusively  verbal 
exchange  was  reduced  because  both  participants  were  viewing  the  same  screen  while  discussing 
the  options  at  that  point  in  the  program.  Only  a  one-way  view  was  available  to  show  the  instructor 
computer  screen  to  remote  students.  Instructor-student  interactions  on  resolving  problems  were 
relatively  brief  in  this  course,  but  verbal  exchanges  resulting  from  not  being  able  to  see  student 
screens  probably  increased  the  time  required  to  resolve  student  problems.  To  show  remote-student 
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computer  screens  to  the  instructor,  laptops  would  have  to  be  carried  to  the  document  camera  or  a 
more  elaborate  configuration  would  be  required  (e.g.,  students  could  take  their  computer  to  a 
centrally  located  scan  converter,  wiring  could  be  extended  to  student  workstations,  or  a  within 
room  video  network  could  be  used).  This  would  better  allow  the  instructor  to  provide  assistance  to 
students,  but  a  hard-wired  configuration  could  create  logistical  wiring  problems  within  a  VTT 
classroom. 

The  use  of  existing  VTT  classrooms  for  a  laboratory  course  requires  training  equipment  that  is 
portable  or  can  be  adapted  to  be  portable.  VTT  classrooms  must  be  used  by  other  courses  and  this 
sharing  means  that  the  room  cannot  be  dedicated  to  the  materials  or  equipment  for  just  one  course. 
Portable  roll-away  training  equipment  has  been  used  beneficially  in  other  VTT  courses  for  fiber 
optic  cable  systems  and  for  demonstration  training  aids  (Simpson,  et  al.,  1992;  Wetzel,  et  al.,  1996). 
Laptop  computers  were  used  for  the  QA  course  because  they  were  portable  and  could  be  taken  in 
and  out  of  the  classrooms  with  little  difficulty.  Additionally,  the  clutter  of  wiring  required  to 
connect  to  a  printer  was  avoided  by  using  a  wireless  local  area  network  (LAN).  Some  problems 
with  printing  documents  in  the  computer  laboratory  were  due  to  the  positioning  of  the  infrared 
LAN  devices  within  the  room.  These  devices  bounce  signals  within  the  room  and  some  absorbent 
materials  on  the  walls  and  ceiling  may  attenuate  communication  with  certain  regions  of  the  room. 
This  problem  was  addressed  by  adjusting  the  position  of  the  server  or  the  student  stations  to 
maintain  reliable  communications  with  all  stations.4 

The  portable  laptop  computer  system  approach  could  be  used  for  other  CESN  computer 
laboratory  courses  if  modifications  to  existing  rooms  cannot  be  accommodated.  Were  their 
sufficient  demand  for  training  using  computers,  permanent  installations  of  more  elaborate 
computer  systems  could  be  developed.  It  would  be  possible  to  free  desktop  space  for  other  classes 
by  using  computer  desks  with  monitors  mounted  within  the  desk  that  can  be  viewed  through  a  glass 
viewport  in  the  desk  top.  However,  the  size  of  these  desks  would  reduce  the  number  of  student  seats 
per  room  compared  with  the  desks  now  used  for  lecture  classes.  A  wired  LAN  could  be  installed 
within  the  room  and  a  video  network  could  be  used  to  select  a  student  computer  screen  to  be  sent 
out  over  the  VTT  system.  A  more  ambitious  computer  laboratory  configuration  could  also  be 
developed.  For  example,  a  data  network  could  be  installed  in  parallel  to  the  VTT  system  to  allow 
local  and  remote  site  computers  to  be  internetworked  (e.g.,  data  could  be  shared  between  sites, 
student  work  could  be  inspected  remotely,  or  remotely  operated  programs  could  be  displayed  on 
higher  resolution  monitors  than  those  used  for  the  VTT  system  video). 

Laboratory  courses  also  place  greater  demands  on  remote  sites  and  VTT  facilitators  (Wetzel, 
1996;  Wetzel,  et  al.,  1996).  VTT  facilitators  play  both  behavioral  and  logistical  roles.  The 
behavioral  role  is  as  an  assistant  and  intermediary  to  the  instructor.  The  facilitator  serves  as  an 
agent  of  the  instructor  within  the  remote  room  and  also  acts  on  behalf  of  the  students  in  redirecting 
their  questions  and  problems  to  the  instructor.  The  facilitator's  logistical  role  in  this  course  would 
require  setting  up  the  VTT  classroom  with  the  portable  course  equipment  and  then  storing  it  away 
following  a  class.  The  facilitator  also  would  be  required  to  support  the  LAN  and  computer  systems. 


4An  alternative  laptop  computer  configuration  could  employ  hard-wired  PCMCIA  local  area  netwoik  cards.  This 
wiring  could  remain  in  place  and  the  benefit  of  being  able  to  remove  the  portable  laptop  computers  when  other 
courses  used  the  classroom  would  still  be  realized. 
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The  success  of  delivering  the  QA  course  was  due  in  part  to  the  various  preparation  efforts  and 
to  the  fact  that  the  laboratory  was  not  excessively  challenging.  The  course  involved  a  mild  form  of 
a  hands-on  laboratory  (Wetzel,  1996)  in  the  sense  that  the  computer  laboratory  is  not  long,  the  task 
is  not  difficult,  and  students  could  perform  the  task  without  extensive  instructor  assistance.  Other 
kinds  of  computer  skill-learning  laboratories  could  involve  more  complex  software  and  lengthy 
laboratories  that  would  be  more  challenging  than  were  those  in  the  present  course. 

Recommendations 

The  following  recommendations  are  for  the  Chief  of  Naval  Education  and  Training  and  the 
CNET  Electronic  Schoolhouse  Network. 

1.  Delivery  of  the  Quality  Assurance  course  by  videoteletraining  has  been  successful  and 
the  use  of  this  method  should  be  expanded  to  other  sites  as  appropriate  to  satisfy  demand  for  the 
course. 

2.  Portable  laptop  computers  should  be  used  if  more  permanent  installations  cannot  be 
made  within  VTT  classrooms  for  this  and  similar  courses. 
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Appendix  A 
Computer  Quiz 


A-0 


FORCES  AFLOAT  QA  CWP  GENERATOR 
QUESTIONNAIRE 


How  is  the  CWP  generator  accessed? 


A)  C:\foxprow\cwpfox\cwpgen 
C)  C:\cd  cwpgen\  cwp 

2)  Why  are  files  merged? 

A)  Edit  CWP 

C)  All  of  the  above 

3)  How  is  a  merged  file  saved? 

A)  Cannot  save  a  merged  file 

C)  Automatically  saved  in  CWP 
generator 


B)  C:\win,  cwp,  cwp  icon 
D)  None  of  the  above 

B)  Print  QA  form 
D)  None  of  the  above 


B)  Change  file  name 
in  WordPerfect 

D)  None  of  the  above 


4)  What  version  of  CWP  generator  is  being  utilized? 

A)  Foxprow  B)  DOS 

C)  Windows  D)  A  and  C 

5)  What  version  of  WordPerfect  is  required  in  order  to  use 
the  CWP  generator. 


A)  5.1 

C)  5.1  or  6.0 


B)  6.0 
D)  None 


6)  How  are  the  merged  files  accessed  in  WordPerfect? 

A)  Ctrl  F  B)  Alt  D 

C)  Shift  G  D)  None  of  the  above 

7)  Changes  made  to  the  CWP  are  save  in? 

A)  CWP  generator  and  WordPerfect  B)  WordPerfect 
C)  CWP  generator  D)  None  of  the  above 
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8)  IF  the  CWP  program  " locks-up"  pressing  Ctrl,  Alt,  Delete 
will  free  the  program  and  allow  the  editing  process  to 
continue . 

A)  True  B)  False 

9)  Help  can  be  called  up  at  the  beginning  of  the  CWP  program 
giving  the  user  instructions  on  how  to  maneuver  within 
the  generator. 

A)  True  B)  False 

10)  Enter,  Ctrl,  Fl,  and  Esc  are  the  primary  keys  utilized 
when  maneuvering  within  the  CWP  generator. 

A)  True  B)  False 


Remarks : 


Appendix  B 
Student  Questionnaire 
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STUDENT  QUESTIONNAIRE  FOR  QUALITY  ASSURANCE  (QA)  COURSE 

The  purpose  of  this  survey  is  to  collect  information  that  will 
help  make  a  decision  about  the  value  of  remote-site  training  by  means 
of  video  teletraining  (VTT) .  The  information  that  you  provide  will  NOT 
be  used  to  evaluate  you  or  your  instructor.  It  will  be  used  only  for 
research  purposes  to  evaluate  VTT  and  will  not  be  kept  in  your  service 
records.  Please  answer  the  questions  as  honestly  and  completely  as  you  can. 

a .  Name _  _  _  _ 

Last  First  MI  Rate  &  Rank 

b.  Today's  date _  c.  Location  [  ]  San  Diego 

[  ]  Pearl  Harbor 

[  ]  _ 


For  each  of  the  statements  followed  by  [1]  [2]  [3]  [4]  [5]  ,  please  circle 

the  box  indicating  how  well  you  agree  with  the  statement  using  this  scale: 


[1] 

Strongly 

Disagree 


[2] 

Disagree 


[3] 

Neither 

Agree/ 

Disagree 


[4] 

Agree 


[5] 

Strongly 

Agree 


INSTRUCTORS 

1.  Instructors  were  prepared  &  presented 

lessons  clearly 

2.  Instructors  encouraged  class  participation 

3 .  Instructors  answered  student  questions  adequately 

4.  Instructors  helped  students  who  needed  assistance 

5 .  Instructors  maintained  adequate  control  of  class 


LEARNING  ACTIVITIES  &  TRAINING  AIDS 

6.  I  could  clearly  see  the  instructional  materials 

presented  by  the  instructor 

7.  I  could  clearly  hear  the  instruction 

8 .  The  course  provided  skills  that  can  be 

applied  on  the  job 

9.  I  had  access  to  the  instructors  outside  of  class 

concerning  the  course  content 

10.  Instruction  and  demonstrations  on  completing  the 

forms  was  adequate 

11.  Instruction  and  demonstration  of  computer 

procedures  was  adequate  for  the  computer  lab 

12 .  Paper  and  pen  exercises  were  completed  without 

problems 


Disagree 
SD  D 

N 

Agree 

A  SA 

ti] 

[2] 

[3] 

[4] 

[5] 

[i] 

[2] 

[3] 

[4] 

[5] 

[i] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

Disagree 
SD  D 

N 

Agree 

A  SA 

ti] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

ti] 

[2] 

[3] 

[4] 

[5] 

[i] 

[2] 

[3] 

[4] 

[5] 
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Disagree  Agree 


SD 

D 

N 

A 

SA 

13. 

The  computer  laboratory  was  accomplished 

[i] 

[2] 

[3] 

[4] 

[5] 

without  problems 

14. 

Training  aids  were  used  effectively 

[i] 

[2] 

[3] 

[4] 

[5] 

15. 

Details  of  training  aids  could  be  clearly  seen 

(for  example,  transparencies/graphics/ forms ) 

[i] 

[2] 

[3] 

[4] 

[5] 

16. 

Table  space  and  room  layout  were  adequate  to 

accommodate  the  use  of  computers 

[i] 

[2] 

[3] 

[4] 

[5] 

INTERACTION/ PARTICIPATION 

Disagree 

Agree 

SD 

D 

N 

A 

SA 

17. 

Interaction  between  instructors  and  students 

was  adequate  to  support  learning  objectives 

[i] 

[2] 

[3] 

[4] 

[5] 

18. 

Student  participation  was  sufficient  to  support 

learning  objectives 

[i] 

[2] 

[3] 

[4] 

[5] 

19. 

Students  aided  one  another  in  performing 

course  exercises 

[i] 

[2] 

[3] 

[4] 

[5] 

20. 

Instructors  gave  equal  attention  to  all 

students 

til 

[2] 

[3] 

[4] 

[5] 

21. 

I  hesitated  to  ask  questions  in  order  to  clarify 

procedures  or  concepts 

[i] 

[2] 

[3] 

[4]  [5] 

22  . 

The  ability  of  students  to  get  help  was  adequate 

[i] 

[2] 

[3] 

[4] 

[5] 

23. 

The  course  provided  skills  that  can  be 

applied  on  the  job 

[i] 

[2] 

[3] 

[4] 

[5] 

EFFECTIVENESS  OF  SPECIFIC  PORTIONS  OF  THE  COURSE 


24.  The  lectures  were  accomplished  with: 

[  ] -No  problems  [  ] -Few  problems  [  ] -Some  problems  [  ] -Many  problems 

25.  The  laboratory  exercises  with  paperwork  were  accomplished  with: 

[  ] -No  problems  [  ] -Few  problems  [  ] -Some  problems  [  ] -Many  problems 

26.  The  computer  laboratory  was  accomplished  with: 

[  ] -No  problems  [  ] -Few  problems  [  ] -Some  problems  [  ] -Many  problems 

27.  Printing  documents  was  accomplished  with: 

[  ] -No  problems  [  ] -Few  problems  [  ] -Some  problems  [  ] -Many  problems 

28.  Operating  the  computer  program  was  accomplished  with: 

[  ] -No  problems  [  ] -Few  problems  [  ] -Some  problems  [  ] -Many  problems 
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29.  When  working  with  the  computer  program: 

[  ]  I  worked  in  a  team  and  someone  else  operated  the  program 

[  ]  I  worked  in  a  team  and  I  operated  the  program  myself 

[  ]  I  worked  in  a  team  and  we  switched  off  operating  the  program 

[  ]  I  worked  alone  and  operated  the  program  myself 


BACKGROUND  Disagree  Agree 

SD  D  N  A  SA 

30.  Before  this  course,  I  had  basic  knowledge  of  [1]  [2]  [3]  [4]  [5] 

Quality  Assurance  concepts  and  forms 

31.  How  would  you  rate  your  general  skill  or  background  in  using  computers: 

[  ]  None /Beginner 
[  ]  Novice 
[  ]  Intermediate 
[  ]  Accomplished 
[  ]  Advanced 

32.  What  is  your  level  of  typing  or  keyboarding  skill? 

[  ]  Hunt-and-Peck 
[  ]  Beginner 
[  ]  Intermediate 
[  ]  Accomplished 
[  ]  Advanced 


VIDEO-TELETRAINING  Disagree  Agree 

SD  D  N  A  SA 

33.  Instructors  coordinated  activities  among  [1]  [2]  [3]  [4]  [5] 

classroom  sites  effectively 

34.  Local  and  remote  site  classrooms 

participated  equally  in  the  class  [1]  [2]  [3]  [4]  [5] 

35.  Instructors  called  on  students  at  all  of  the 

classroom  sites  about  equally  [1]  [2]  [3]  [4]  [5] 

36.  The  VTT  facilitators  assisting  the  instructor  [1]  [2]  [3]  [4]  [5] 

were  helpful  to  students  in  the  remote  class 

37.  Who  most  frequently  provided  assistance  to  you: 

[  ]  Instructor 
[  ]  VTT  Facilitator 
[  ]  Other  students 
[  ]  No  help  needed 

[  ]  Other  (please  explain) :  _ 

38.  Which  method  of  instruction  would  you  prefer  for  this  course: 

[  ]  Video  Tele-Training  (VTT) 

[  ]  Traditional  method  (non-VTT) 

[  ]  Either  method 

39.  How  did  the  VTT  method  of  instruction  affect  your  opportunities  to 

interact  with  the  instructor? 

[  ]  more  opportunities 
[  ]  no  effect  on  opportunities 
[  ]  fewer  opportunities 
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40.  Which  of  the  following  would  you  prefer?  (check  one) 

[  ]  Enrolling  in  a  VTT  course  near  your  home  port? 

[  ]  Enrolling  in  a  traditional  (live)  course  farther  (TAD)  from 
your  home  port? 

41.  Differences  in  time  zones  between  San  Diego  and  the  Remote  site  were: 

[  ]  No  problem 
[  ]  Mild  problem 
[  ]  Severe  problem 

42.  Compared  to  offering  the  course  by  a  traditional  method, 

offering  this  course  by  VTT  results  in: 

[  ]  More  problems  than  traditional  method 
[  ]  No  effect  on  problems  encountered 
[  ]  Fewer  Problems  than  traditional  method 

43.  If  you  had  a  choice,  would  you  take  another  VTT  course? 

[  ]  Yes 
[  ]  No 

Please  explain: _ 


44.  What  did  you  like  the  most  about  this  course?. 


45.  What  did  you  like  the  least  about  this  course?. 


46.  Discuss  problems  encountered  during  the  laboratories. 


47 .  Discuss  any  suggestions  you  have  for  improving  the  course. 


Appendix  C 

Interaction  Tally  Categories  and  Summary  Statistics 
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VTT  Interaction  Tally  Categories 

An  observer  recorded  the  frequency  of  interactions  that  occurred  over  the  VTT 
network  in  several  categories  given  below.  Interactions  had  to  be  related  to  course 
content.  Start  and  end  times  for  each  class  period  were  recorded  on  the  tally  form  in 
order  to  account  for  variations  of  these  periods  over  different  class  convenings.  For  each 
of  the  first  five  categories  below,  a  check  mark  was  entered  in  a  box  corresponding  to 
whether  it  occurred  at  the  local  site  or  the  remote  site. 

Instructor  questions  (Categories  1-3): 

(1)  Instructor  question  open  to  any  site:  Instructor  asks  a  question  (and  receives 
an  answer)  in  which  no  site  is  identified  so  that  all  sites  are  free  to  respond  to  the 
question. 

(2)  Instructor  question  names  a  site:  Instructor  asks  a  question  (and  receives  an 
answer)  that  identifies  a  site  which  should  answer,  either  the  local  site,  a  specific 
remote  site,  or  "any"  remote  site. 

(3)  Instructor  questions  specific  student:  Instructor  asks  a  question  (and  receives 
an  answer)  directed  at  one  student  by  singling  out  the  student  by  name,  title,  or  by 
pointing  (a  class  roster  for  each  site  was  created  during  the  first  morning  of  the 
class). 

Student  questions  (Categories  4-5): 

(4)  Student  initiated  question:  A  student  initiates  a  question  to  the  instructor, 
class,  or  another  student. 

(5)  Student  conversations  and  exchanges:  Student  exchanges,  comments,  or 
general  conversations  involving  continued  back-and-forth  exchanges  on  an  issue, 
such  as  to  resolve  a  problem.  These  were  most  commonly  a  continuation  of  a 
student  initiated  question  that  resulted  in  more  than  one  communication  each  for 
the  instructor  and  student,  including  another  student  than  the  one  initiating  the 
original  question. 

Unanswered  questions  and  microphone  reminders  (Categories  6-8): 

Several  other  miscellaneous  circumstances  were  recorded  separately  in  addition  to 
the  above  categories.  These  were  recorded  for  the  whole  class  and  not  by  remote 
and  local  sites,  or  only  for  local  students  in  the  case  of  category  8: 

(6)  No  answer  to  question:  There  was  No  Answer  from  students  to  an  instructor 
question  which  was  specific  to  the  instructional  content. 

(7)  No  answer  to  generic  request  for  questions:  The  instructor  asked  in  one  way 
or  another  if  there  were  "Any  Questions"  and  no  response  was  received  from  the 
students.  These  questions  were  generic  queries  that  were  used  to  pause  before 
proceeding  with  the  instruction. 

(8)  Reminders  to  local  students  to  use  microphones:  The  instructor  reminded 
the  local  San  Diego  site  to  use  the  microphones  so  remote  sites  could  hear 
questions  and  comments. 
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Table  C-l 

Symbols  and  Formulas  Used  in  Interaction  Rate  Computations 


I 

Sum  of  number  of  interactions  over  four  classes 

H 

= 

Total  recording  hours  over  four  classes 

C 

- 

Number  of  classes 

S 

= 

Total  number  of  students  over  four  classes 

S/C 

= 

Students  per  class 

H/C 

= 

Recording  hours  per  class 

I/H 

= 

Interactions  per  hour 

(I/H)/(S/C) 

= 

Interactions  per  hour  per  student  (per  class) 

(I/H)/(S/C) 

(I/S)/(H/C)  =  [(I/C)/(H/C)]/(S/C) 

Table  C-2 

Instructor  and  Student  Interaction  Frequencies  for  Class  Activities 


Lectures 

Paper  Labs 

Computer  Lab 

Measure  and  Group 

Local 

Remote 

Local 

Remote 

Local 

Remote 

Number  of  students  (S) 

48 

53 

48 

52.5 

_ 

38 

Number  of  classes  (C) 

4 

4 

4 

4 

- 

3 

Students  per  class  (S/C) 

12 

13.25 

12 

13.12 

- 

12.66 

Total  recording  hours  (H) 

18.26 

18.26 

9.33 

9.33 

- 

5.30 

Recording  hours  per  class  (H/C) 

4.56 

4.56 

2.33 

2.33 

1.76 

Interaction  Frequencies  (I) 

Instructor  Questions: 

(1)  Open  to  any  site 

109 

73 

5 

2 

3 

(2)  Names  a  site 

94 

101 

2 

7 

- 

3 

(3)  Specific  student 

25 

23 

2 

0 

- 

4 

Student  Questions: 

(4)  Student  initiated 

32 

18 

46 

12 

7 

(5)  Conversations/exchanges 

31 

18 

14 

4 

- 

0 

All  instructor  (1-3) 

228 

197 

9 

9 

_ 

10 

All  student  (4-5) 

63 

36 

60 

16 

- 

7 

All  instructor  and  student  (1-5) 

291 

233 

69 

25 

- 

17 

Note:  Lectures  and  paper  laboratories  were  during  first  two  days  of  class.  Computer 
laboratory  was  on  third  day  of  class  (only  remote  students  were  in  VTT  classroom)  and 
only  three  classes  were  observed  due  to  equipment  failure.  Letters  in  parentheses  denote 
measures  given  in  Table  C-l.  Numbers  in  parentheses  are  interaction  tally  category 
numbers  defined  at  start  of  appendix. 
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Table  C-3 

Instructor  and  Student  Interaction  Rates 


Lectures 

Paper  Labs 

Computer  Lab 

Measure  and  Group 

Local 

Remote 

Local 

Remote 

Local 

Remote 

Interactions  Per  Hour  (I/H) 

Instructor  Questions: 

(1)  Open  to  any  site 

5.96 

3.99 

0.53 

0.21 

- 

0.56 

(2)  Names  a  site 

5.14 

5.52 

0.21 

0.75 

- 

0.56 

(3)  Specific  student 

1.36 

1.25 

0.21 

0.00 

- 

0.75 

Student  Questions: 

(4)  Student  initiated 

1.75 

0.98 

4.92 

1.28 

- 

1.32 

(5)  Conversations/exchanges 

1.69 

0.98 

1.50 

0.42 

- 

0.00 

All  instructor  (1-3) 

12.48 

10.78 

0.96 

0.96 

- 

1.88 

All  student  (4-5) 

3.44 

1.97 

6.42 

1.71 

- 

1.32 

All  instructor  and  student  (1-5)  15.93  12.75 

Interactions  Per  Hour  Per  Student  (I/H)/(S/C) 

7.39 

2.67 

3.20 

Instructor  Questions: 

(1)  Open  to  any  site 

0.49 

0.30 

0.04 

0.01 

- 

0.04 

(2)  Names  a  site 

0.42 

0.41 

0.01 

0.05 

- 

0.04 

(3)  Specific  student 

0.11 

0.09 

0.01 

0.00 

- 

0.05 

Student  Questions: 

(4)  Student  initiated 

0.14 

0.07 

0.41 

0.09 

- 

0.10 

(5)  Conversations/exchanges 

0.14 

0.07 

0.12 

0.03 

- 

0.00 

All  instructor  (1-3) 

1.04 

0.81 

0.08 

0.07 

- 

0.14 

All  student  (4-5) 

0.28 

0.14 

0.53 

0.13 

- 

0.10 

All  instructor  and  student  (1-5) 

1.32 

0.96 

0.61 

0.20 

- 

0.25 

Note:  Lectures  and  paper  laboratories  were  during  first  two  days  of  class.  Computer 
laboratory  was  on  third  day  of  class  (only  remote  students  were  in  VTT  classroom)  and 
only  3  classes  were  observed  due  to  equipment  failure.  All  values  in  table  are  truncated 
from  greater  precision. 
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Table  C-4 

Interaction  Frequencies  and  Rates  for 
Unanswered  Questions  and  Microphone  Reminders 


Measure  and  Category  Lectures 

Paper 

Labs 

Computer 

Lab 

Interaction  Frequencies  (I) 

(6)  No  answer  to  instructor  question 

Open  to  any  site 

3 

0 

0 

Names  a  site  or  student: 

Local 

2 

0 

Remote 

7 

1 

0 

(7)  No  answer  to  "any  questions"  query 

47 

9 

2 

(8)  Microphone  reminders  to  local  students 

12 

2 

— 

Interactions  Per  Hour  (I/H) 

(6)  No  answer  to  instructor  question 

Open  to  any  site 

0.164 

0.000 

0.000 

Names  a  site  or  student: 

Local 

0.109 

0.000 

Remote 

0.383 

0.107 

0.000 

(7)  No  answer  to  "any  questions"  query 

2.572 

0.964 

0.377 

(8)  Microphone  reminders  to  local  students 

0.656 

0.214 

— 

Interactions  Per  Hour  Per  Student  (I/H)/(S/C) 

(6)  No  answer  to  instructor  question 

Open  to  any  site 

0.013 

0.000 

0.000 

Names  a  site  or  student: 

Local 

0.009 

0.000 

Remote 

0.028 

0.008 

0.000 

(7)  No  answer  to  "any  questions"  query 

0.213 

0.073 

0.029 

(8)  Microphone  reminders  to  local  students 

0.054 

0.017 

— 

Note:  Only  remote  students  were  present  in  the  VTT  classroom  during  the  computer 
laboratory.  Microphones  reminders  for  category  (8)  are  for  local  classroom  only. 
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Table  C-5 

Interaction  Frequencies  and  Rates  for  Lectures  on  First  Two  Days  of  Class 


Measure  and  Category 

Day  1 

Day  2 

Difference 

Total  Recording  Hours  (H)  Per  Day 

11.116 

7.150 

Interaction  Frequencies  (I) 

Instructor  Questions  (1-3) 

Local 

121 

107 

Remote 

100 

97 

Student  Questions  (4-5) 

Local 

39 

24 

Remote 

19 

17 

Unanswered  instructor  questions  (6) 

6 

6 

No  answer  to  "any  questions"  query  (7) 

23 

24 

Microphone  reminders  to  local  students  (8) 

9 

3 

Interactions  Per  Hour  (I/H) 

Instructor  Questions  (1-3) 

Local 

10.884 

14.965 

4.081 

Remote 

8.995 

13.566 

4.571 

Student  Questions  (4-5) 

Local 

3.508 

3.356 

-0.152 

Remote 

1.709 

2.377 

0.668 

Unanswered  instructor  questions  (6) 

0.539 

0.839 

0.299 

No  answer  to  "any  questions"  query  (7) 

2.068 

3.356 

1.287 

Microphone  reminders  to  local  students  (8) 

0.809 

0.419 

-0.390 

Interactions  Per  Hour  Per  Student  (I/H)/(S/C) 

Instructor  Questions  (1-3) 

Local 

0.907 

1.247 

0.340 

Remote 

0.678 

1.023 

0.345 

Student  Questions  (4-5) 

Local 

0.293 

0.279 

-0.014 

Remote 

0.128 

0.179 

0.051 

Unanswered  instructor  questions  (6) 

0.021 

0.033 

0.012 

No  answer  to  "any  questions"  query  (7) 

0.081 

0.132 

0.050 

Microphone  reminders  to  local  students  (8) 

0.067 

0.034 

-0.033 

Note:  Interaction  rates  adjust  for  unequal  recording  periods  that  are  reflected  in  interaction 
frequencies.  Only  the  lecture  class  activity  is  compared  over  days.  Recording  time  was  too  brief 
to  compare  paper  laboratories  because  only  a  half  an  hour  period  was  given  on  the  first  day  (total 
hours  for  the  four  classes  were  1 .92  on  day  1  and  7.42  on  day  2). 
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Table  C-6 

Correlation  Between  Observer  Interaction  Tallies 


Interaction  Category 

Number  of 
Observation 
Pairs 

Correlation 
Simulated 
Remote  Classes 

Correlation 

Actual 

Remote  Classes 

(1)  Instructor  Question  Open  to  Any  Site 

8 

0.876 

0.952 

(2)  Instructor  Question  Names  a  Site 

8 

0.520 

0.835 

(3)  Instructor  Questions  Specific  Student 

8 

0.484 

0.769 

(4)  Student  Initiated  Questions 

8 

0.489 

0.648 

(5)  Student  Conversations  and  Exchanges 

8 

0.663 

0.557 

All  5  Categories  Combined: 

Local  Site  Only 

20 

0.868 

0.930 

Remote  Site  Only 

20 

0.889 

0.906 

Local  and  Remote  Combined 

40 

0.870 

0.917 

Explanation  of  Table  C-6:  Rater  agreement  using  the  interaction  tally  was  determined 
during  lectures  on  the  first  and  second  day  of  class.  Data  were  obtained  from  two  dry-run 
class  convenings  when  the  remote  classroom  condition  was  simulated  in  San  Diego  and 
during  two  class  convenings  with  actual  remote  students  in  Pearl  Harbor.  Pearson 
correlation  coefficients  were  calculated  for  two  raters  observing  the  same  sessions  in 
which  there  was  one  local  and  one  remote  site  (perfect  correspondence  between  raters 
would  be  indicated  by  a  coefficient  of  +1.0).  The  overall  0.87  (simulated  remote)  or 
0.917  (actual  remote)  correlation  between  rater  interaction  frequencies  was  based  on  an 
overall  total  of  40  pairs  of  observations  where  each  rater  made  observations  for  five  tally 
categories  (1-5)  by  two  sites  (local  and  remote)  by  two  recording  periods  (first  and 
second  day  of  class)  by  two  separate  class  convenings.  The  two  correlations  for  only 
local  or  remote  sites  were  each  based  on  20  observations  (i.e.,  two  recording  periods  by 
two  class  convenings  by  five  tally  categories).  The  five  correlations  for  the  individual 
categories  were  based  on  eight  observations  in  each  category  (i.e.,  two  sites  by  two 
recording  periods  by  two  class  convenings  for  each  rater).  The  total  interactions 
recorded  during  the  two  simulated  remote  class  convenings  was  239  for  one  rater  and 
223  for  the  other  rater;  during  the  two  actual  remote  class  convenings  these  sums  were 
262  and  274.  Only  one  rater  observed  all  four  of  the  actual  remote  site  classes  to  Pearl 
Harbor  and  only  that  rater’s  data  was  used  in  the  analyses  given  in  previous  tables. 

The  interpretation  of  one  series  of  interactions  posed  a  scoring  difficulty  for  raters. 
The  instructor  required  successive  individual  students  to  read  one  statement  from  a  list  of 
statements  in  a  student  manual.  An  individual  student  was  to  read  one  statement  and 
then  the  next  student  was  to  read  the  next  statement  in  the  series.  This  sequence 
sometimes  alternated  between  sites  or  among  students  at  one  site.  In  the  midst  of  this 
series,  an  interaction  became  an  implicit  question  because  each  student  was  not  verbally 
designated  to  read  a  statement.  A  discrepancy  between  rater  interpretations  of  this 
situation  was  resolved  by  defining  these  circumstances  to  be  questions  that  implicitly 
named  a  specific  student  (category  3)  rather  than  ones  that  named  a  site  (category  2). 
The  table  shows  that  the  correlation  between  raters  improved  in  the  two  later  classes 
(actual  remote)  as  a  consequence  of  resolving  this  discrepancy. 
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Appendix  D 

Student  Summary  Statistics 


D-0 


Table  D-l 


Number  and  Percent  of 

Enlisted  and  Officer  Students  by  Treatment  Group 


Officer  and  Enlisted  Student  Subgroups 


Treatment  Group 


Officer  Enlisted  All  Students 


VTT  Local  . 
VTT  Remote 
Traditional 


5  (10.42%) 
4  (7.69%) 
17  (12.78%) 


43  (89.58%) 
48  (92.31%) 
116  (87.21%) 


48  (100%) 
52  (100%) 
133  (100%) 


Combined  Groups 


26  (11.16%)  207  (88.84%)  233  (100%) 


Note:  Percentages  are  within  rows  of  total  students  per  treatment  group.  The  proportion 
of  officer  and  enlisted  students  by  treatment  group  did  not  differ  significantly  by  a  test  of 
proportions  expressed  as  Chi  Square,  %2(2)=1.01,  p>.05  (Fleiss,  1981,  p.139). 


Table  D-2 

Average  Military  Rank  for 
Enlisted  and  Officer  Students  by  Treatment  Group 


Group 

Mean 

Military  Rank 

Standard 

Deviation 

Range 

(Min-Max) 

Number  of 
Students 

Officer: 

VTT  Local 

1.60 

0.89 

1-3 

5 

VTT  Remote 

1.25 

0.50 

1-2 

4 

Traditional 

2.00 

1.17 

1-4 

17 

All  Officer  Groups 

1.81 

1.05 

1-4 

26 

Enlisted: 

VTT  Local 

5.48 

1.54 

1-8 

43 

VTT  Remote 

5.77 

1.01 

4-9 

48 

Traditional 

6.12 

1.09 

4-9 

116 

All  Enlisted  Groups 

5.91 

1.21 

1-9 

207 

Note:  Min-Max  for  range  is  the  minimum  and  maximum  military  rank. 
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Table  D-3 


Average  Examination  Percent  Correct 
by  Treatment  group  for  Officer  and  Enlisted  Students 


Mean 

Standard 

Range 

Number  of 

Group 

Exam  Percent 

Deviation 

(Min-Max) 

Students 

Officer: 

VTT  Local 

97.20 

2.28 

94-100 

5 

VTT  Remote 

94.00 

4.89 

88-100 

4 

Traditional 

95.64 

4.70 

82-100 

17 

All  Officer 

95.69 

4.33 

82-100 

26 

Enlisted: 

VTT  Local 

93.41 

4.77 

80-100 

43 

VTT  Remote 

91.75 

6.69 

74-100 

48 

Traditional 

93.61 

6.78 

56-100 

116 

All  Enlisted 

93.14 

6.41 

56-100 

207 

Combined  Officer-Enlisted: 

All  Local 

93.81 

4.71 

80-100 

48 

All  Remote 

91.92 

6.56 

74-100 

52 

All  Traditional 

93.87 

6.57 

56-100 

133 

All  Groups 

93.42 

6.26 

56-100 

233 

Adjusted  Means: 

All  Local 

94.13 

All  Remote 

92.14 

All  Traditional 

93.66 

Note:  Min-Max  for  range  is  the  minimum  and  maximum  examination  percentage  correct. 
The  correlation  of  final  grade  percent  correct  with  z-score  military  rank  for  all  three 
treatment  groups  combined  was  r=0.1495  (df=231,  p=0.02).  The  correlation  of  officer- 
enlisted  status  with  final  grade  percent  correct  for  all  three  treatment  groups  combined 
was  r=0.1285  (df=231,  p=0.0501).  The  adjusted  means  shown  in  the  table  are  based  on 
an  analysis  of  covariance  (ANCOVA)  using  the  z-score  military  rank  and  officer-enlisted 
status  as  covariates.  The  combination  of  these  covariates  was  significantly  related  to 
examination  performance,  F(2,228)=4.22,  p=.015.  However,  adjusting  the  small 
differences  between  treatment  groups  for  these  covariates  does  not  change  the  finding 
that  the  treatment  groups  did  not  differ  significantly,  ANCOVA  F(2,228)=1.55,  p>.05. 
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Table  D-4 

Percentage  of  Students  Passing  Course 


Group 

Number  of 
Students 

Number 

Failing 

Number 

Passing 

Percent 

Passing 

VTT  Local 

48 

48 

100.0  % 

VTT  Remote 

52 

1 

51 

98.1  % 

Traditional 

133 

5 

128 

96.2  % 

Combined  Groups 

233 

6 

227 

97.4  % 

Note:  Passing  for  the  final  examination  was  80%  and  above.  The  proportion  of  students 
passing  in  the  three  treatment  groups  was  not  significantly  different,  %2(2)=2.09,  p>.05 
(Fleiss,  1981,  p.139). 


Table  D-5 

Average  Percent  Correct  on  Computer  Quiz 


Group 

Mean 

Percent  Correct 

Standard 

Deviation 

Range 

(Min-Max) 

Number  of 
Students 

VTT  Local 

46.53 

18.55 

0-90 

49 

VTT  Remote 

45.75 

16.39 

0-80 

47 

Combined  Groups 

46.15 

17.43 

0-90 

96 

Note:  Min-Max  for  range  is  the  minimum  and  maximum  quiz  percentage  correct. 
Percent  correct  is  based  on  10  possible  correct  items  for  quiz  shown  in  Appendix  A. 


D-3 


Appendix  E 

Computer  Equipment  and  Cost  Analysis  Tables 


E-0 


Table  E-l 

Computer  System  Equipment  Used  in  Evaluation  Study 


Local  Transmitting  Site 


Instructor  Demonstration  Laptop  Computer: 

IBM  360  Laptop  Computer  (486SX/33Mhz;  4Mb  RAM;  170Mb  Harddrive)  $2,100 
4  Mb  RAM  upgrade  to  total  of  8Mb  $210 

Included  Software:  PCDOS  V6.3;  Windows  for  Workgroups  V3. 1 1  $0 

Total:  $2,310 

Scan  Converter  (Computer  VGA  to  NTSC  video):  $  1 ,600 

Remote  Site 


Student  Laptop  Computer: 

IBM  360  Laptop  Computer  (486SX/33Mhz;  4Mb  RAM;  170Mb  Harddrive)  $2,100 
4  Mb  RAM  upgrade  to  total  of  8Mb  $210 

Photonix  PCMCIA  Infrared  Local  Area  Network  (LAN)  Adapter  $450 

Included  Software:  PCDOS  V6.3;  Windows  for  Workgroups  V3.1 1  $0 

IBM  TCP/IP  for  DOS/Windows  V2. 1 . 1  Software  $  1 59 

IBM  TCP/IP  NFS  Kit  for  DOS/Windows  V2. 1 . 1  Software  $94 

Total  One  System  (10  systems  were  used  in  the  research):  $3,013 

Print  Server  Computer: 

IBM  Compatible  PC  Computer  (486/66Mhz;  16Mb  RAM;  420Mb  Hardrive)  $1,350 
CDROM  Drive  $150 

IBM  Infrared  Local  Area  Network  (LAN)  Adapter  $400 

Computer  Monitor  $300 

OS2  WARP  multitasking  operating  software  $  1 15 

IBM  TCP/IP  for  OS2  V2.0  Software  $  1 39 

IBM  TCP/IP  NSF  Kit  for  OS2  Software  $  1 06 

IBM  TCP/IP  DOS/Windows  Access  Kit  for  OS2  V2.0  Software  $62 

Total:  $2,622 

Server  Hewlett-Packard  LaserJet  4+  Printer:  $  1 ,375 

Miscellaneous: 

Flat  Cord  Guard  Power  Strips  $250 

Portable  cart  with  lockable  doors  for  storing  computers  $500 


Note:  There  is  no  implied  endorsement  for  any  of  the  commercial  products  mentioned  in  this 
report.  In  most  cases  there  are  alternative  products  that  could  have  been  employed  and 
mention  of  these  products  simply  documents  the  actual  equipment  used  in  the  research. 
Product  names  and  brands  mentioned  herein  are  trademarks  of  their  respective  holders. 
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Table  E-2 

Estimates  for  Computer  System  Equipment  Used  in  Cost  Analysis 


Local  Transmitting  Site: 

1  Instructor  Demonstration  Laptop  Computer 

1  Scan  Converter 

$2,310 

$1,600 

Total 

$3,910 

5  Computer  Remote  Site: 

5  Student  Laptop  Computers  (@  $3,013  each) 

1  Print  Server  Computer 

1  Laser  Printer 

$15,065 

$2,622 

$1,375 

Total 

$19,062 

10  Computer  Remote  Site: 

10  Student  Laptop  Computers  (@  $3,013  each) 

1  Print  Server  Computer 

1  Laser  Printer 

$30,130 

$2,622 

$1,375 

Total 

$34,127 

1  Remote  +  1  Local 

2  Remotes  +  1  Local 

Steps  of  Computation  5  Computers 

10  Computers 

5  Computers 

10  Computers 

Local  Site 

$3,910 

$3,910 

$3,910 

$3,910 

1st  Remote  Site 

$19,062 

$34,127 

$19,062 

$34,127 

2nd  Remote  Site 

— 

— 

$19,062 

$34,127 

Total  Equipment 

$22,972 

$38,037 

$42,034 

$72,164 

Maintenance  for  5  years 
(10%  Equipment  per  year) 

$11,486 

$19,019 

$21,017 

$36,082 

Total  Equipment  +  Maintenance 

$34,458 

$57,056 

$63,051 

$108,246 

Yearly  Cost  of  Computer  System 
(Amortized  Over  5  Year  Period) 
(Total  /  5  years) 

$6,892 

$11,411 

$12,610 

$21,649 

Cost  Per  Class  Convening 

Amortized  Within  a  Year 
(Yearly  Cost  /  Number  Convenings) 

1  Convening 

$6,892 

$11,411 

$12,610 

$21,649 

4  Convenings 

$1,723 

$2,853 

$3,153 

$5,412 

8  Convenings 

$  862 

$1,426 

$1,576 

$2,706 

12  Convenings 

$  574 

$  951 

$1,051 

$1,804 
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Table  E-3 

VTT  Classroom  Costs  and  Computer  System  Costs 


VTT  Classroom  Costs: 

VTT  Classroom  Contract  Cost  (1  room  per  year)* *  $70,000 

Available  Training  Days  a  Year  25 1 

VTT  Classroom  Cost  Per  Day  (1  classroom)  $279 

VTT  Classroom  Cost  Per  Class  Convening  (3  days,  1  classroom)  $837 

One  Class  Convening  Cost  (3  days,  2  or  3  classrooms): 

2  Classrooms  (1  local,  1  remote)  $1,674 

3  Classrooms  (1  local,  2  remote)  $2,51 1 

Combined  Yearly  Cost  of  VTT  Classrooms  and  Computer  Equipment** 

For  Different  Numbers  of  Class  Convenings  Per  Year: 

2  VTT  Classrooms  ( 1  Remote  +  1  Local) 

5  Computers  at  1  Remote  Site: 

4  Convenings  x  ($1,674  VTT  +  $1,723  Computer)  =  $13,588 

8  Convenings  x  ($1,674  VTT  +  $862  Computer)  =  $20,288 

12  Convenings  x  ($1,674  VTT  +  $574  Computer)  =  $26,976 

10  Computers  at  1  Remote  Site: 

4  Convenings  x  ($1,674  VTT  +  $2,853  Computer)  =  $18,108 

8  Convenings  x  ($1,674  VTT  +  $1,426  Computer)  =  $24,800 

12  Convenings  x  ($1,674  VTT  +  $951  Computer)  =  $31,500 

3  VTT  Classrooms  (2  Remotes  +  1  Local) 

5  Computers  per  Each  of  2  Remote  Sites: 

4  Convenings  x  ($2,51 1  VTT  +  $3,153  Computer)  =  $22,656 

8  Convenings  x  ($2,51 1  VTT  +  $1,576  Computer)  =  $32,696 

12  Convenings  x  ($2,51 1  VTT  +  $1,051  Computer)  =  $42,744 

10  Computers  per  Each  of  2  Remote  Sites: 

4  Convenings  x  ($2,51 1  VTT  +  $5,412  Computer)  =  $31,692 

8  Convenings  x  ($2,511  VTT  +  $2,706  Computer)  =  $41,736 

12  Convenings  x  ($2,51 1  VTT  +  $1,804  Computer)  =  $51,780 


*Note:  VTT  classroom  contract  costs  based  on  average  of  all  CESN  classrooms. 

**Note:  Amortized  computer  equipment  costs  per  convening  from  bottom  of  Table  E-2. 
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Table  E-4 

Avoided  Student  Travel  Costs  for  One  Class  Convening 


Per  Student  Avoided  Travel  Costs  by  Participating  Sites 
1  Remote  +  1  Local  Site  2  Remote  +  1  Local  Sites 


Individual  Site 

PH-SD 

BA-SD 

SF-SD 

PH-BA-SD 

PH-SF-SD 

BA-SF-SD 

Pearl  Harbor  (PH) 

608 

- 

- 

608 

608 

- 

Bangor  WA  (BG) 

- 

346 

- 

346 

- 

346 

San  Francisco  (SF) 

- 

- 

204 

- 

204 

204 

San  Diego  (SD) 

0 

0 

0 

0 

0 

0 

Total  All  Sites: 

(1  student  each  site) 

$608 

$346 

$204 

$954 

$812 

$550 

Number  Students 

Total  Avoided  Costs  for  All  Remote  Sites  (Equal  Students  Per  Site) 

Each  Remote  Site 

PH-SD 

BA-SD 

SF-SD 

PH-BA-SD 

PH-SF-SD 

BA-SF-SD 

1 

608 

346 

204 

954 

812 

550 

2 

1,216 

692 

408 

1,908 

1,624 

1,100 

3 

1,824 

1,038 

612 

2,862 

2,436 

1,650 

4 

2,432 

1,384 

816 

3,816 

3,248 

2,200 

5 

3,040 

1,730 

1,020 

4,770 

4,060 

2,750 

6 

3,648 

2,076 

1,224 

5,724 

4,872 

3,300 

7 

4,256 

2,422 

1,428 

6,678 

5,684 

3,850 

8 

4,864 

2,768 

1,632 

7,632 

6,496 

4,400 

9 

5,472 

3,114 

1,836 

8,586 

7,308 

4,950 

10 

6,080 

3,460 

2,040 

9,540 

8,120 

5,500 

11 

6,688 

3,806 

2,244 

10,494 

8,932 

6,050 

12 

7,296 

4,152 

2,448 

11,448 

9,744 

6,600 

13 

7,904 

4,498 

2,652 

12,402 

10,556 

7,150 

14 

8,512 

4,844 

2,856 

13,356 

11,368 

7,700 

15 

9,120 

5,190 

3,060 

14,310 

12,180 

8,250 

16 

9,728 

5,536 

3,264 

15,264 

12,992 

8,800 

17 

10,336 

5,882 

3,468 

16,218 

13,804 

9,350 

18 

10,944 

6,228 

3,672 

17,172 

14,616 

9,900 

19 

11,552 

6,574 

3,876 

18,126 

15,428 

10,450 

20 

12,160 

6,920 

4,080 

19,080 

16,240 

11,000 

Note:  Entries  in  the  table  assume  an  equal  number  of  students  at  all  remote  sites  for  the  three 
day  course.  These  per  student  costs  are  based  on  the  sum  of  airfare,  local  travel,  and  per 
diem  costs.  Government  rate  round-trip  airfare  to  San  Diego  from  Pearl  Harbor,  Bangor,  and 
San  Francisco  was  $480,  $218,  and  $76  respectively.  Local  travel  was  estimated  to  be  $20. 
Per  diem  for  three  training  days  and  one  travel  day  was  $108  (based  on  a  $27  composite  rate 
adjusted  appropriate  to  1  in  10  students  being  an  officer  at  a  slightly  higher  rate  of  $45 
compared  with  $25  for  the  enlisted  rate). 
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Table  E-5 

Avoided  Student  Travel  Costs  in  Excess  of  VTT  Classroom  Costs 


Participating  Sites 

Students  Per 

No.  Class 

1  Remote  +  1  Local 

2  Remotes  +  1  Local 

Remote  Site 

Convenings 

PH-SD 

BA-SD 

SF-SD 

PH-BA-SD 

PH-SF-SD 

BA-SF-SD 

10 

4 

17,624 

7,144 

1,464 

28,116 

22,436 

11,956 

8 

35,248 

14,288 

2,928 

56,232 

44,872 

23,912 

12 

52,872 

21,432 

4,392 

84,348 

67,308 

35,868 

15 

4 

29,784 

14,064 

5,544 

47,196 

38,676 

22,956 

8 

59,568 

28,128 

11,088 

94,392 

77,352 

45,912 

12 

89,352 

42,192 

16,632 

141,588 

116,028 

68,868 

20 

4 

41,944 

20,984 

9,624 

66,276 

54,916 

33,956 

8 

83,888 

41,968 

19,248 

132,552 

109,832 

67,912 

12 

125,832 

62,952 

28,872 

198,828 

164,748 

101,868 

Note:  Table  entries  are  avoided  student  travel  costs  minus  VTT  classroom  costs.  Avoided 
student  travel  costs  are  from  Table  E-4  and  are  multiplied  by  the  number  of  convenings. 
VTT  classroom  costs  from  top  of  Table  E-3  are  for  two  rooms  ($1,674)  or  three  rooms 
($2,511)  and  are  multiplied  by  the  number  of  class  convenings.  Abbreviations  for  sites 
participating  in  a  class  are:  San  Diego  (SD)  hub  site  originating  instruction  and  remote  sites 
at  Pearl  Harbor  (PH),  Treasure  Island,  San  Francisco  (SF),  and  Bangor,  Washington  (BA). 
Only  remote  students  generate  cost  avoidance,  but  costs  for  a  VTT  classroom  at  the 
transmitting  hub  site  at  San  Diego  are  included  with  costs  for  remote  site  classrooms. 


E-5 


Table  E-6 


Avoided  Student  Travel  Costs  in  Excess  of  Combined  Costs  for 
VTT  Classrooms  and  Computer  Equipment  (5  or  10  per  site) 


Students  Per  No.  Class 

Remote  Site  Convenings  _ Participating  Sites  and  Computers  Per  Site 


1  Remote  +  1  Local  2  Remotes  +  1  Local 

5  computers  at  1  remote  10  computers  (5  each  remote) 


PH-SD 

BA-SD 

SF-SD 

PH-BA-SD 

PH-SF-SD 

BA-SF-SD 

10 

4 

10,732 

252 

-5,428 

15,504 

9,824 

-656 

8 

28,352 

7,392 

-3,968 

43,624 

32,264 

11,304 

12 

45,984 

14,544 

-2,496 

71,736 

54,696 

23,256 

15 

4 

22,892 

7,172 

-1,348 

34,584 

26,064 

10,344 

8 

52,672 

21,232 

4,192 

81,784 

64,744 

33,304 

12 

82,464 

35,304 

9,744 

128,976 

103,416 

56,256 

20 

4 

35,052 

14,092 

2,732 

53,664 

42,304 

21,344 

8 

76,992 

35,072 

12,352 

119,944 

97,224 

55,304 

12 

118,944 

56,064 

21,984 

186,216 

152,136 

89,256 

1  Remote  +  1  Local 

2  Remotes  +  1  Local 

10  computers  at  1  remote 

20  computers  (10  each  remote) 

PH-SD 

BA-SD 

SF-SD 

PH-BA-SD 

PH-SF-SD 

BA-SF-SD 

10 

4 

6,212 

-4,268 

-9,948 

6,468 

788 

-9,692 

8 

23,840 

2,880 

-8,480 

34,584 

23,224 

2,264 

12 

41,460 

10,020 

-7,020 

62,700 

45,660 

14,220 

15 

4 

18,372 

2,652 

-5,868 

25,548 

17,028 

1,308 

8 

48,160 

16,720 

-320 

72,744 

55,704 

24,264 

12 

77,940 

30,780 

5,220 

119,940 

94,380 

47,220 

20 

4 

30,532 

9,572 

-1,788 

44,628 

33,268 

12,308 

8 

72,480 

30,560 

7,840 

110,904 

88,184 

46,264 

12 

1 14,420 

51,540 

17,460 

177,180 

143,100 

80,220 

Note:  Table  entries  are  student  travel  costs  (values  in  Table  E-4  times  the  number  of  convenings) 
minus  the  sum  of  VTT  room  costs  and  amortized  computer  equipment  costs  (Table  E-3).  Computer 
costs  are  for  remote  sites  only.  With  5  computers  per  site,  two  students  per  computer  are 
accommodated  at  10  students  and  a  double-shift  laboratory  is  required  for  15  and  20  students.  With 
10  computer  per  site,  one  student  per  computer  is  accommodated  at  10  students,  half  the  students 
share  computers  at  15  students,  and  all  students  are  two  per  computer  at  20  students.  Additional 
convenings  could  represent  more  than  one  course  sharing  the  use  of  the  same  computers.  Site 
abbreviations  as  per  previous  tables. 
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Appendix  F 

Student  Questionnaire  Summary  Tables 
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Table  F-l 


Means,  Standard  Deviations,  and  Analysis  of  Variance  Results 
for  Student  Questionnaire  Rating  Items. 


Question 

Mean 

Local  Remote 

Std.  Dev. 

Local  Remote 

ANOVA 

df  F 

INSTRUCTORS 

1 .  Instructors  were  prepared  &  presented 
lessons  clearly 

4.40 

4.27 

0.54 

0.60 

1,98 

1.24 

2.  Instructors  encouraged  class  participation 

4.56 

4.54 

0.50 

0.61 

1,98 

0.05 

3.  Instructors  answered  student  questions 
adequately 

4.33 

4.23 

0.72 

0.78 

1,98 

0.46 

4.  Instructors  helped  students  who  needed 
assistance 

4.48 

4.21 

0.55 

0.82 

1,98 

3.60 

5.  Instructors  maintained  adequate  control 
of  class 

4.60 

4.33 

0.49 

0.68 

1,97 

5.05* 

LEARNING  ACTIVITIES  &  TRAINING  AIDS 

6.  I  could  clearly  see  the  instructional 
materials  presented  by  the  instructor 

4.56 

4.10 

0.58 

0.77 

1,98 

* 

* 

00 

7.  I  could  clearly  hear  the  instruction 

4.62 

4.48 

0.53 

0.54 

1,98 

1.80 

8.  The  course  provided  skills  that  can  be 
applied  on  the  job 

4.35 

4.21 

0.67 

0.82 

1,98 

0.89 

9.  I  had  access  to  the  instructors  outside  of 
class  concerning  the  course  content 

3.96 

3.47 

0.87 

1.08 

1,97 

6.03* 

10.  Instruction  and  demonstrations  on 
completing  the  forms  was  adequate 

4.17 

4.04 

0.81 

0.87 

1,97 

0.57 

1 1 .  Instruction  &  demonstration  of  computer 
procedures  was  adequate  for  computer  lab 

4.08 

3.77 

0.79 

1.13 

1,98 

2.54 

12.  Paper  and  pen  exercises  were  completed 
without  problems 

4.08 

4.17 

0.74 

0.68 

1,98 

0.40 

13.  The  computer  laboratory  was  accomplished 
without  problems 

4.11 

3.31 

0.79 

1.23 

1,97 

14.47** 

14.  Training  aids  were  used  effectively 

4.44 

4.25 

0.50 

0.68 

1,98 

2.42 

15.  Details  of  training  aids  could  be  clearly  seen 
(for  example,  transparencies/graphics/forms) 

4.56 

4.06 

0.50 

0.70 

1,98 

17.01** 

16.  Table  space  and  room  layout  were  adequate 
to  accommodate  the  use  of  computers 

4.38 

4.48 

0.70 

0.54 

1,98 

0.72 

*p<.05  **p<.01 

Five  point  scale  for  Items  #1-23,  #30,  and  #33-36  was: 

[1]  Strongly  Disagree  [2]  Disagree  [3]  Neither  Agree/Disagree  [4]  Agree  [5]  Strongly  Agree 
Four  point  scale  for  Items  #24-28  was: 

[l]-No  problems  [2] -Few  problems  [3]-Some  problems  [4] -Many  problems 
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Table  F-l  Continued 


Question 

Mean 

Local  Remote 

Std.  Dev. 

Local  Remote 

ANOVA 

df  F 

INTERACTION/PARTICIPATION 

17.  Interaction  between  instructors  and  students 
was  adequate  to  support  learning  objectives 

4.36 

4.25 

0.53 

0.65 

1,97 

0.86 

18.  Student  participation  was  sufficient  to 
support  learning  objectives 

4.35 

4.35 

0.60 

0.52 

1,98 

0.01 

19.  Students  aided  one  another  in  performing 
course  exercises 

4.33 

4.35 

0.56 

0.56 

1,98 

0.01 

20.  Instructors  gave  equal  attention  to  all 
students 

4.48 

4.10 

0.55 

0.89 

1,98 

6.58* 

21. 1  hesitated  to  ask  questions  in  order 
to  clarify  procedures  or  concepts 

2.25 

2.49 

1.16 

1.07 

1,97 

1.16 

22.  The  ability  of  students  to  get  help 
was  adequate 

4.43 

4.10 

0.54 

0.80 

1,97 

5.65* 

23.  The  course  provided  skills  that  can  be 
applied  on  the  job 

4.21 

4.14 

0.62 

0.93 

1,98 

0.21 

EFFECTIVENESS  OF  PORTIONS  OF  COURSE 
24.  Lectures  were  accomplished  with: 

1.52 

1.83 

0.65 

0.81 

1,98 

4.29* 

25.  Laboratory  exercises  with  paperwork 
were  accomplished  with: 

1.65 

1.81 

0.63 

0.77 

1,98 

1.31 

26.  Computer  laboratory  was  accomplished  with: 

1.61 

2.29 

0.62 

0.94 

1,92 

16.76** 

27.  Printing  documents  was  accomplished  with: 

1.38 

2.12 

0.57 

1.00 

1,76 

11.83** 

28.  Operating  the  computer  program  was 
accomplished  with: 

1.55 

2.13 

0.55 

0.89 

1,92 

14.04** 

BACKGROUND 

30.  Before  this  course,  I  had  basic  knowledge 
of  Quality  Assurance  concepts  and  forms 

2.81 

3.12 

1.39 

1.14 

1,93 

1.47 

VIDEO-TELETRAINING 

33.  Instructors  coordinated  activities  among 
classroom  sites  effectively 

4.23 

4.24 

0.66 

0.56 

1,96 

0.01 

34.  Local  and  remote  site  classrooms 
participated  equally  in  the  class 

4.12 

4.18 

0.81 

0.72 

1,96 

0.13 

35.  Instructors  called  on  students  at  all 
of  the  classroom  sites  about  equally 

4.50 

4.30 

0.50 

0.54 

1,96 

3.55 

36.  The  VTT  facilitators  assisting  the  instructor 
were  helpful  to  students  in  the  remote  class 

4.02 

4.10 

0.79 

0.89 

1,92 

0.20 

All  items  were  rated  on  a  five  point  scale  except  for  Items  #  24-28  where  a  four  point  scale  was  used: 
[l]-No  problems  [2]-Few  problems  [3]-Some  problems  [4]-Many  problems 
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Table  F-2 

Percent  Response  for  Computer  Background  Questionnaire  Items  31  and  32 


Question  and  Answer  Category 

Treatment  Group 

VTT  Local 

VTT  Remote  Combined  Groups 

Number  of  students  responding 

48 

51 

99 

31.  How  would  you  rate  your 

general  skill  or  background 

in  using  computers: 

None/Beginner 

14.6 

27.5 

21.2 

Novice 

22.9 

19.6 

21.2 

Intermediate 

37.5 

39.2 

38.4 

Accomplished 

16.7 

9.8 

13.1 

Advanced 

8.3 

3.9 

6.1 

Total  Percent 

100.0 

100.0 

100.0 

32.  What  is  your  level  of  typing 

or  keyboarding  skill: 

Hunt-and-Peck 

18.8 

15.7 

17.2 

Beginner 

27.1 

35.3 

31.3 

Intermediate 

41.7 

23.5 

32.3 

Accomplished 

10.4 

21.6 

16.2 

Advanced 

2.1 

3.9 

3.0 

Total  Percent 

100.0 

100.0 

100.0 

Note:  Percents  exclude  students  who  did  not  answer  (one  in  each  group). 
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Table  F-3 


Percentage  Response  Rate  of  Students  in  Each  Treatment  Group 
That  Offered  Comments  to  Open-Ended  Questions 
(Items  43, 44, 45, 46, 47,  and  Computer  Quiz) 


Treatment  Group 

Question  and  Group 

VTT  Local 

VTT  Remote  Combined  Groups 

43.  Take  another  VTT  course 

54.2 

53.8 

54.0 

44.  Liked  most  about  course 

72.9 

59.6 

66.0 

45.  Liked  least  about  course 

50.0 

51.9 

51.0 

46.  Problems  during  laboratories 

22.9 

48.1 

36.0 

47.  Suggestions  for  improvement 

39.6 

36.5 

38.0 

CWP  Computer  Quiz  "Remarks" 

14.3 

38.3 

26.0 

Note:  percentages  of  total  students  that  gave  one  or  more  comments  to  a  question. 
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Table  F-4 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question  43 
"If  you  had  a  choice  would  you  take  another  VTT  course?;  Please  explain" 


Group  and  Comment  Category 

Local 

Treatment  Group 

Remote 

Combined 

a.  VTT  benefits 

20.0 

17.5 

18.7 

b.  Interaction 

8.0 

7.0 

7.5 

c.  Miscellaneous  positive  toward  VTT 

2.0 

14.0 

8.4 

a-c.  Subtotal  Positive  Comments 

30.0 

38.5 

34.6 

d.  Prefer  traditional  instruction 

10.0 

5.3 

7.5 

e.  VTT  problems 

8.0 

5.3 

6.5 

f.  Miscellaneous  negative  comments 

8.0 

8.8 

8.4 

d-f.  Subtotal  Negative  Comments 

26.0 

19.4 

22.4 

No  Comment 

44.0 

42.1 

43.0 

Total  Percent 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

26 

28 

54 

Number  of  2nd  Comments 

2 

4 

6 

Number  of  3rd  Comments 

0 

1 

1 

Total  Comments 

28 

33 

61 

Number  No  Comment 

22 

24 

46 

Total  Comments  and  No  Comment 

50 

57 

107 

Number  of  Students 

48 

52 

100 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment. 

a.  VTT  Benefits:  save  travel,  save  money,  expand  training. 

b.  Interaction:  positive  comments  on  interacting  with  other  students  and  locations  by  VTT. 

c.  Miscellaneous  Positive  toward  VTT:  VTT  is  good,  liked  good  training,  enjoyed  VTT. 

d.  Prefer  traditional  instruction,  live/local  instruction. 

e.  VTT  problems:  Equipment/procedure  problems  with  VTT  system. 

f.  Miscellaneous  negative  comments:  VTT  confusing,  need  better  facilitators, 

didn’t  like  course,  can’t  ask  questions,  don’t  like  being  watched,  need  larger  class. 
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Table  F-5 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question  44 
"What  did  you  like  the  most  about  this  course?" 


Treatment  Group 


Group  and  Comment  Category 

Local 

Remote 

Combined 

a.  Novelty  of  VTT  instruction 

26.4 

23.1 

24.8 

b.  Computer  and  CWP  hands-on  training 

24.5 

21.2 

22.8 

c.  Learning  about  Quality  Assurance 

20.8 

11.5 

16.2 

d.  Instructors 

3.8 

3.8 

3.8 

No  Comment 

24.5 

40.4 

32.4 

Total  Percent 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

35 

31 

66 

Number  of  2nd  Comments 

5 

0 

5 

Total  Comments 

40 

31 

71 

Number  No  Comment 

13 

21 

34 

Total  Comments  and  No  Comment 

53 

52 

105 

Number  of  Students 

48 

52 

100 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment. 

a.  Novelty  of  YTT  instruction:  interacting  with  other  site,  discussions,  interesting, 
being  part  of  new  instruction,  stay  at  one  site,  save  money, 

time  schedule  was  adjusted  for  both  sites. 

b.  Computer  and  CWP  hands-on  Training:  liked  computer  training,  good  labs, 
generating  a  CWP,  hands-on  exercises. 

c.  Learning  about  Quality  Assurance:  liked  layout  of  materials,  benefit  to  job, 
applies  to  job,  made  aware  of  safety. 

d.  Instructors:  liked  instructors. 
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Table  F-6 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question  45 
"What  did  you  like  the  least  about  this  course?" 


Treatment  Group 


Group  and  Comment  Category 

Local 

Remote 

Combined 

a.  Course  materials 

18.0 

3.8 

10.7 

b.  CWP  computer  program  laboratory 

4.0 

7.6 

5.8 

c.  Instructors  or  assistance 

6.0 

11.3 

8.7 

d.  VTT  problems 

10.0 

7.5 

8.7 

e.  Waiting  on  other  site 

8.0 

1.9 

4.9 

f.  Classroom  too  hot 

2.0 

11.3 

6.8 

g.  Class  too  early  in  day 

0.0 

9.4 

4.9 

h.  Liked  the  course 

4.0 

0.0 

1.9 

No  Comment 

48.0 

47.2 

47.6 

Total  Percent 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

24 

27 

51 

Number  of  2nd  Comments 

1 

1 

2 

Number  of  3rd  Comments 

1 

0 

1 

Total  Comments 

26 

28 

54 

Number  No  Comment 

24 

25 

49 

Total  Comments  and  No  Comment 

50 

53 

103 

Number  of  Students 

48 

52 

100 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment. 

a.  Course  Materials:  Need  more  content,  too  much  material,  didn’t  learn  anything  new, 
vague,  boring,  looking  in  publications,  jumping  around  manuals. 

b.  CWP  computer  program  laboratory. 

c.  Instructors  or  assistance:  Lack  of  help  from  instructors,  not  enough  instructors, 
instructors  read  too  much,  materials  not  ready. 

d.  VTT  problems:  VTT  limitations,  equipment  problems,  microphones, 
distracting  monitor,  being  on  camera. 

e.  Waiting  on  other  site:  Waiting  for  other  site  to  finish  work. 

f.  Classroom  too  hot:  air  conditioning  was  broken,  too  hot  in  room. 

g.  Class  too  early  in  day. 

h.  Liked  the  course:  positive  rather  than  negative  comment. 
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Table  F-7 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question  46 
"Discuss  problems  encountered  during  the  laboratories" 


Treatment  Group 


Group  and  Comment  Category 

Local 

Remote 

Combined 

a.  Problems  using  CWP  computer  program 

2.1 

23.6 

13.6 

b.  Computer  link  to  printer 

0.0 

7.3 

3.9> 

c.  Need  more  computers 

2.1 

1.8 

1.9 

d.  More  instruction  or  laboratory  time 

10.4 

1.8 

5.8 

e.  VTT  problems 

6.3 

5.5 

5.8 

f.  Negative  comment  on  assistance 

0.0 

5.5 

2.9 

g.  Positive  comment  on  assistance 

2.1 

5.5 

3.9 

No  Comment 

77.1 

49.1 

62.1 

Total  Percent 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

11 

25 

36 

Number  of  2nd  Comments 

0 

3 

3 

Total  Comments 

11 

28 

39 

Number  No  Comment 

37 

27 

64 

Total  Comments  and  No  Comment 

48 

55 

103 

Number  of  Students 

48 

52 

100 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment. 

a.  Problems  using  CWP  computer  program. 

b.  Computer  link  to  printer  caused  problem. 

c.  Need  more  computers. 

d.  More  instruction  or  laboratory  time:  need  more  instruction,  not  enough  time  on  computers, 
not  enough  time  on  laboratory  paper  work,  not  familiar  with  publications  or  terminology. 

e.  VTT  problems:  VTT  equipment  problems  or  communication  line  down. 

f.  Negative  comment  on  assistance:  facilitator  not  present  or  knowledgeable, 

laboratory  papers  not  available. 

g.  Positive  comment  on  assistance:  instructor  or  facilitator  helped, 
working  together  helped,  few  problems  encountered. 
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Table  F-8 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question  47 
"Discuss  any  suggestions  you  have  for  improving  the  course" 


Treatment  Group 


Group  and  Comment  Category 

Local 

Remote 

Combined 

a.  Improve  content  or  delivery 

24.5 

11.1 

17.8 

b.  Improve  CWP  computer  program 

0.0 

14.8 

7.5 

c.  Resolve  equipment  problems 

7.6 

9.3 

8.4 

d.  Improve  facilitator  assistance 

0.0 

3.7 

1.9 

e.  Prefer  traditional  over  VTT  instruction 

3.8 

0.0 

1.9 

f.  Positive  comment  on  instructor,  course,  or  VTT 

9.4 

0.0 

4.7 

No  Comment 

54.7 

61.1 

57.9 

Total  Percent 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

19 

19 

38 

Number  of  2nd  Comments 

4 

2 

6 

Number  of  3rd  Comments 

1 

0 

1 

Total  Comments 

24 

21 

45 

Number  No  Comment 

29 

33 

62 

Total  Comments  and  No  Comment 

53 

54 

107 

Number  of  Students 

48 

52 

100 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment. 

a.  Improve  content  or  delivery:  improve  content,  better  or  more  materials, 
improve  instructor  time  schedule  and  procedures,  have  materials  ready, 
more  hands-on,  give  students  copy  of  CWP  program  to  take  to  ship. 

b.  Improve  CWP  computer  program:  improve  software,  make  program  easier. 

c.  Resolve  equipment  problems:  VTT  equipment,  speakers  or  microphone  quality, 
show  video  of  local  students. 

d.  Improve  facilitator  assistance:  more  involved  facilitator. 

e.  Prefer  traditional  over  VTT  instruction. 

f.  Positive  comment  on  instructor,  course,  or  VTT:  Good  course  or  instructor, 

expand  use  of  VTT  to  deliver  instruction. 
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Table  F-9 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question 
for  "Remarks"  on  CWP  Generator  Computer  Quiz/Questionnaire 


Treatment  Group 


Group  and  Comment  Category 

Local 

Remote 

Combined 

a.  Software  issues 

0.0 

28.8 

14.4 

b.  Positive  comment  on  course  or  computer  program 

17.3 

7.7 

12.5 

c.  Miscellaneous  dissatisfied 

1.9 

7.7 

4.8 

No  Comment 

80.8 

55.8 

68.3 

Total  Percent 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

7 

18 

25 

Number  of  2nd  Comments 

2 

4 

6 

Number  of  3rd  Comments 

1 

1 

2 

Total  Comments 

10 

23 

33 

Number  No  Comment 

42 

29 

71 

Total  Comments  and  No  Comment 

52 

52 

104 

Number  of  Students 

49 

47 

96 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment. 

a.  Software  issues:  Software  not  friendly,  hard  to  print  CWP,  want  to  use  WordPerfect, 
needs  file  merging,  don’t  have  computer  experience,  need  more  training  on  CWP  program, 
need  user  guide  for  program. 

b.  Positive  comment  on  course  or  computer  program:  good  course,  good  or  useful  computer 
program,  want  program  for  ship. 

c.  Miscellaneous  dissatisfied:  instruction  too  short,  VTT  connection  went  down  during  lab. 
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